2023-12-09T00:58:28.465 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-09T00:58:28.466 INFO:teuthology.results:subset: '111/120000'
2023-12-09T00:58:28.466 INFO:teuthology.results:seed: '8320'
2023-12-09T03:11:47.491 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-09T03:11:47.509 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-12-09T15:07:47.034 INFO:teuthology.results:Tests finished! gathering results...
2023-12-09T15:07:59.976 INFO:scrape:Found 159 jobs
2023-12-09T15:08:08.477 INFO:scrape:Found 6 distinct failure reasons
2023-12-09T15:08:08.477 INFO:scrape:Failure: Command failed on smithi131 with status 1: 'sudo yum -y install ceph'
2023-12-09T15:08:08.478 INFO:scrape:21 jobs: ['7484579', '7484680', '7484638', '7484631', '7484590', '7484589', '7484651', '7484707', '7484715', '7484580', '7484701', '7484716', '7484637', '7484710', '7484697', '7484643', '7484567', '7484646', '7484698', '7484582', '7484613']
2023-12-09T15:08:08.478 INFO:scrape:suites intersection: []
2023-12-09T15:08:08.478 INFO:scrape:suites union: ['1-start', '2-node-mgr', '2-thrash', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'agent/off', 'agent/on', 'fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8}', 'orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8', 'orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8', 'orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'orchestrator_cli}', 'root}', 'start', 'task/test_adoption}', 'task/test_ca_signed_key}', 'task/test_extra_daemon_features}', 'task/test_host_drain}', 'task/test_orch_cli_mon}', 'task/test_orch_cli}', 'task/test_rgw_multisite}', 'task/test_set_mon_crush_locations}', 'tasks/rados_api_tests}', 'tasks/rotate-keys}']
2023-12-09T15:08:08.478 INFO:scrape:
2023-12-09T15:08:08.478 INFO:scrape:Failure: Command failed on smithi023 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9eab968a-963b-11ee-95a3-87774f69a715 -e sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2023-12-09T15:08:08.478 INFO:scrape:17 jobs: ['7484649', '7484629', '7484674', '7484679', '7484695', '7484603', '7484615', '7484571', '7484648', '7484662', '7484621', '7484628', '7484599', '7484690', '7484704', '7484585', '7484575']
2023-12-09T15:08:08.479 INFO:scrape:suites intersection: []
2023-12-09T15:08:08.479 INFO:scrape:suites union: ['1-bootstrap/17.2.0', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-config-upgrade/{fail_fs}', '4-final}', '4-verify}', '4-wait', '5-upgrade-ls', '5-upgrade-with-workload', '6-verify}}', 'agent/off', 'agent/on', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/classic}', 'mon_election/connectivity}', 'orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy']
2023-12-09T15:08:08.479 INFO:scrape:
2023-12-09T15:08:08.479 INFO:scrape:Failure: Command failed on smithi032 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest'
2023-12-09T15:08:08.479 INFO:scrape:2 jobs: ['7484647', '7484568']
2023-12-09T15:08:08.480 INFO:scrape:suites intersection: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v1only', 'start', 'tasks/rados_python}']
2023-12-09T15:08:08.480 INFO:scrape:suites union: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v1only', 'orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8', 'start', 'tasks/rados_python}']
2023-12-09T15:08:08.480 INFO:scrape:
2023-12-09T15:08:08.480 INFO:scrape:Dead: b'2023-12-09T15:04:10.410 DEBUG:teuthology.exit:Finished running handlers'
2023-12-09T15:08:08.480 INFO:scrape:1 jobs: ['7484708']
2023-12-09T15:08:08.480 INFO:scrape:suites: ['1-bootstrap/17.2.0', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools']
2023-12-09T15:08:08.480 INFO:scrape:
2023-12-09T15:08:08.480 INFO:scrape:Failure: Command failed on smithi053 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"
2023-12-09T15:08:08.481 INFO:scrape:4 jobs: ['7484705', '7484626', '7484665', '7484586']
2023-12-09T15:08:08.481 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'orch/rook/smoke/{0-distro/ubuntu_20.04']
2023-12-09T15:08:08.481 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}']
2023-12-09T15:08:08.481 INFO:scrape:
2023-12-09T15:08:08.481 INFO:scrape:Failure: Command failed on smithi111 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9dcc13bb80dcc1cdea72b04d80505b4649a2b882 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04cf99d2-9643-11ee-95a3-87774f69a715 -- ceph rgw realm bootstrap -i -'
2023-12-09T15:08:08.481 INFO:scrape:2 jobs: ['7484717', '7484570']
2023-12-09T15:08:08.481 INFO:scrape:suites intersection: ['agent/off', 'mon_election/connectivity', 'task/test_rgw_multisite}']
2023-12-09T15:08:08.481 INFO:scrape:suites union: ['agent/off', 'mon_election/connectivity', 'orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'orch/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_rgw_multisite}']
2023-12-09T15:08:08.482 INFO:scrape:
2023-12-09T15:08:08.487 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/
failed:  46
dead:    1
running: 0
waiting: 0
queued:  0
passed:  112


Fail
=================================================================
[7484567]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:12:58
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484567/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484567/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bdb81ca679f345b0a954e7a8b57fb846

    Command failed on smithi177 with status 1: 'sudo yum -y install ceph'

[7484568]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:12:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484568/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484568/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ac88c96368d34a6691a371a12b7a667e

    Command failed on smithi032 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel
    librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-
    fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest
    python3-pytest'

[7484570]  orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:18:43
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484570/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484570/
sentry: https://sentry.ceph.com/organizations/ceph/?query=028c18fe0a9e46168177def07084923b

    Command failed on smithi031 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:9dcc13bb80dcc1cdea72b04d80505b4649a2b882 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    8e6dafa0-9632-11ee-95a3-87774f69a715 -- ceph rgw realm bootstrap -i -'

[7484571]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:30:53
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484571/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484571/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cbb2f9c54f0b427ba9f82fe15e280cc0

    Command failed on smithi003 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    6639cae6-9632-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484575]  orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic}
-----------------------------------------------------------------
time:   00:43:08
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484575/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484575/
sentry: https://sentry.ceph.com/organizations/ceph/?query=919426109b6849ea886914827f04cc09

    Command failed on smithi116 with status 22: "sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    2748353e-9632-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c 'ceph orch upgrade
    start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.foo'"

[7484579]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:11:05
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484579/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484579/
sentry: https://sentry.ceph.com/organizations/ceph/?query=273c681eb0aa4c2dbb45cb750f59dcbd

    Command failed on smithi131 with status 1: 'sudo yum -y install ceph'

[7484580]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:10:58
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484580/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484580/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e9c0d8ee8b2b48f5b0da292991496e71

    Command failed on smithi129 with status 1: 'sudo yum -y install ceph'

[7484582]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_adoption}
-----------------------------------------------------------------
time:   00:11:15
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484582/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484582/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f93f2ed454a6453396baa5059cc72cec

    Command failed on smithi125 with status 1: 'sudo yum -y install ceph'

[7484585]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:30:46
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484585/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484585/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3d189c97d87d475e8d868758049ae2cf

    Command failed on smithi016 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    c414c664-9634-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484586]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master}
-----------------------------------------------------------------
time:   00:06:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484586/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484586/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e1653be736474561b69e4f845503b1a4

    Command failed on smithi177 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7484589]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_ca_signed_key}
-----------------------------------------------------------------
time:   00:11:22
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484589/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484589/
sentry: https://sentry.ceph.com/organizations/ceph/?query=269008affac84940b3dc99274d4ceeea

    Command failed on smithi153 with status 1: 'sudo yum -y install ceph'

[7484590]  orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/on orchestrator_cli}
-----------------------------------------------------------------
time:   00:11:33
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484590/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484590/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2cef9c56950e4321a109f2bfe3f2b8a3

    Command failed on smithi037 with status 1: 'sudo yum -y install ceph'

[7484599]  orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
-----------------------------------------------------------------
time:   00:36:00
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484599/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484599/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5885b7928e1f49e59dae0e4b1acd97df

    Command failed on smithi067 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    6db93fb0-9635-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484603]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:39:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484603/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484603/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3ce2d6d8aece43749ec24b4bb98c53c1

    Command failed on smithi066 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    cdc4df7a-9637-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484613]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_host_drain}
-----------------------------------------------------------------
time:   00:11:49
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484613/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484613/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b2664c1c726d4d51920ae35acc5f74c0

    Command failed on smithi037 with status 1: 'sudo yum -y install ceph'

[7484615]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:38:19
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484615/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484615/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a77d6b1770b14f7dbe9c3f701e975698

    Command failed on smithi132 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    2fe49474-9639-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484621]  orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic}
-----------------------------------------------------------------
time:   00:49:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484621/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484621/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bf984a21c8ce42b3824e6c723e2fef9b

    Command failed on smithi005 with status 22: "sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    becee1c2-9638-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c 'ceph orch upgrade
    start --image quay.ceph.io/ceph-ci/ceph:$sha1 --services rgw.foo'"

[7484626]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484626/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484626/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0f8d71d9d05349ea95aee1f0c44a8e1f

    Command failed on smithi142 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7484628]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:33:56
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484628/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484628/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9e4316aee6a24a8a89d8cd5a7854348e

    Command failed on smithi088 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    70b26980-963a-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484629]  orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:33:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484629/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484629/
sentry: https://sentry.ceph.com/organizations/ceph/?query=806cee490b4e44bc85694eddcbcc3908

    Command failed on smithi027 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    5b7743f2-9639-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484631]  orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/off orchestrator_cli}
-----------------------------------------------------------------
time:   00:11:44
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484631/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484631/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c496905e87604df999fab8416482acde

    Command failed on smithi023 with status 1: 'sudo yum -y install ceph'

[7484637]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:11:27
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484637/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484637/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ff378e4dcf14497ca764959bcd979b36

    Command failed on smithi148 with status 1: 'sudo yum -y install ceph'

[7484638]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:11:41
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484638/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484638/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eea0a7997a4c4c879ffe77668a92a74e

    Command failed on smithi037 with status 1: 'sudo yum -y install ceph'

[7484643]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:11:37
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484643/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484643/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d97810d6f5464c588e055d8e33546a6e

    Command failed on smithi142 with status 1: 'sudo yum -y install ceph'

[7484646]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
-----------------------------------------------------------------
time:   00:11:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484646/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484646/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b0f2c5bd686246908f64a5f6d8075a3a

    Command failed on smithi118 with status 1: 'sudo yum -y install ceph'

[7484647]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
-----------------------------------------------------------------
time:   00:10:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484647/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484647/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e4e0c0e730db49609cfb5fd2d6b40e1c

    Command failed on smithi032 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel
    librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-
    fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest
    python3-pytest'

[7484648]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:33:58
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484648/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484648/
sentry: https://sentry.ceph.com/organizations/ceph/?query=288bcc76a9334117b04094f9e3163c34

    Command failed on smithi135 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    ae59d6fe-963c-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484649]  orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
-----------------------------------------------------------------
time:   00:35:00
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484649/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484649/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0222283ca5d14a31ac63ffb29f84b40b

    Command failed on smithi023 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    9eab968a-963b-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484651]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_set_mon_crush_locations}
-----------------------------------------------------------------
time:   00:11:19
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484651/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484651/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0ffc75976e5140afb1adc71cbd81663c

    Command failed on smithi125 with status 1: 'sudo yum -y install ceph'

[7484662]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:33:57
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484662/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484662/
sentry: https://sentry.ceph.com/organizations/ceph/?query=76bd61da46a749a18ecaa6d04d0add6e

    Command failed on smithi104 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    dda80870-963e-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484665]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master}
-----------------------------------------------------------------
time:   00:06:31
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484665/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484665/
sentry: https://sentry.ceph.com/organizations/ceph/?query=efc8941c1c014090b017080179a9920f

    Command failed on smithi001 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7484674]  orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
-----------------------------------------------------------------
time:   00:31:11
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484674/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484674/
sentry: https://sentry.ceph.com/organizations/ceph/?query=472f6fe1d843480688ee58701383fc08

    Command failed on smithi067 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    29317ad8-963f-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484679]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:41:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484679/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484679/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d00f126cdee8408eb64860158ddb2fa7

    Command failed on smithi006 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    9cd64198-9640-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484680]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_extra_daemon_features}
-----------------------------------------------------------------
time:   00:11:17
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484680/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484680/
sentry: https://sentry.ceph.com/organizations/ceph/?query=122e0ed2db52435d8bcf30155558f045

    Command failed on smithi086 with status 1: 'sudo yum -y install ceph'

[7484690]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:14:35
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484690/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484690/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cb1fc890c57d48d6b96e490e948c30df

    Command failed on smithi044 with status 127: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    a19df756-9641-11ee-95a3-87774f69a715 -- ceph orch host add smithi149'

[7484695]  orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
-----------------------------------------------------------------
time:   00:51:16
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484695/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484695/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3dc9d80820ea4be284a2043f9de5f509

    Command failed on smithi062 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    f218ca1e-963f-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph orch
    upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date |
    length == 7\'"\'"\'\''

[7484697]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root}
-----------------------------------------------------------------
time:   00:11:47
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484697/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484697/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c2c05811808e46e5a1a3ce5645108e55

    Command failed on smithi184 with status 1: 'sudo yum -y install ceph'

[7484698]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys}
-----------------------------------------------------------------
time:   00:12:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484698/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484698/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4bcfdb72a228474f8bddd70d8258d6a5

    Command failed on smithi052 with status 1: 'sudo yum -y install ceph'

[7484701]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_orch_cli}
-----------------------------------------------------------------
time:   00:12:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484701/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484701/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2eb84bd96ca745b297bb02998570b1ba

    Command failed on smithi162 with status 1: 'sudo yum -y install ceph'

[7484704]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:30:47
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484704/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484704/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6b33325b3cc540e4a47674e21ab7a948

    Command failed on smithi061 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    016e3a24-9642-11ee-95a3-87774f69a715 -e
    sha1=9dcc13bb80dcc1cdea72b04d80505b4649a2b882 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7484705]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:29
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484705/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484705/
sentry: https://sentry.ceph.com/organizations/ceph/?query=16794b0064624c619f7de9abf6b8f7a1

    Command failed on smithi053 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7484707]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:12:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484707/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484707/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c7de3eb845df4282a263309f10ccc477

    Command failed on smithi139 with status 1: 'sudo yum -y install ceph'

[7484710]  orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli}
-----------------------------------------------------------------
time:   00:11:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484710/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484710/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f2c4d38afd3a47029adb3b9711124a72

    Command failed on smithi146 with status 1: 'sudo yum -y install ceph'

[7484715]  orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
-----------------------------------------------------------------
time:   00:11:16
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484715/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484715/
sentry: https://sentry.ceph.com/organizations/ceph/?query=89c2ac26cfac43ef82a8ec2db8b2b8aa

    Command failed on smithi172 with status 1: 'sudo yum -y install ceph'

[7484716]  orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:11:06
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484716/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484716/
sentry: https://sentry.ceph.com/organizations/ceph/?query=448d2586490e4601869d48fcc180d203

    Command failed on smithi190 with status 1: 'sudo yum -y install ceph'

[7484717]  orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:18:32
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484717/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484717/
sentry: https://sentry.ceph.com/organizations/ceph/?query=70fbc75cfd9d4482ba8f7c64b6c61e60

    Command failed on smithi111 with status 22: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:9dcc13bb80dcc1cdea72b04d80505b4649a2b882 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    04cf99d2-9643-11ee-95a3-87774f69a715 -- ceph rgw realm bootstrap -i -'



Dead
=================================================================
[7484708]  orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484708/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484708/

    hit max job timeout



Pass
=================================================================
[7484562] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:25:32
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484562/

[7484563] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:22:11
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484563/

[7484564] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:14:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484564/

[7484565] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:20:35
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484565/

[7484566] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_orch_cli_mon}
time:   00:38:13
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484566/

[7484569] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:23:41
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484569/

[7484572] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:16:25
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484572/

[7484573] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:17:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484573/

[7484574] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:14:44
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484574/

[7484576] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:19:07
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484576/

[7484577] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:16:03
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484577/

[7484578] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_set_mon_crush_locations}
time:   00:19:53
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484578/

[7484581] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:19:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484581/

[7484583] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:15:15
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484583/

[7484584] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:18:07
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484584/

[7484587] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:18:11
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484587/

[7484588] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nvmeof 3-final}
time:   00:19:49
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484588/

[7484591] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:17:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484591/

[7484592] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/rgw 3-final}
time:   00:16:21
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484592/

[7484593] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:14:26
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484593/

[7484594] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
time:   01:02:16
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484594/

[7484595] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests}
time:   00:44:47
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484595/

[7484596] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_cephadm}
time:   00:18:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484596/

[7484597] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:19:29
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484597/

[7484598] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:20:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484598/

[7484600] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:17:49
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484600/

[7484601] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:19:06
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484601/

[7484602] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_cephadm_repos}
time:   00:09:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484602/

[7484604] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:17:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484604/

[7484605] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:43:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484605/

[7484606] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python}
time:   00:24:44
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484606/

[7484607] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_extra_daemon_features}
time:   00:31:25
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484607/

[7484608] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:18:58
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484608/

[7484609] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:19:32
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484609/

[7484610] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/jaeger 3-final}
time:   00:16:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484610/

[7484611] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:26:47
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484611/

[7484612] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:20:17
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484612/

[7484614] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:15:27
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484614/

[7484616] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:48:31
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484616/

[7484617] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys}
time:   00:37:05
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484617/

[7484618] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:17:45
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484618/

[7484619] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:23:55
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484619/

[7484620] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
time:   00:20:59
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484620/

[7484622] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:18:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484622/

[7484623] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_monitoring_stack_basic}
time:   00:25:01
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484623/

[7484624] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:20:18
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484624/

[7484625] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:21:44
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484625/

[7484627] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_orch_cli}
time:   00:19:42
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484627/

[7484630] orch/cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs}
time:   00:46:04
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484630/

[7484632] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:56:35
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484632/

[7484633] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:29:01
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484633/

[7484634] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:21:30
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484634/

[7484635] orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final}
time:   00:18:40
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484635/

[7484636] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:27:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484636/

[7484639] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:16:48
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484639/

[7484640] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_orch_cli_mon}
time:   00:50:20
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484640/

[7484641] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:31:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484641/

[7484642] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:19:31
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484642/

[7484644] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:24:20
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484644/

[7484645] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:19:07
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484645/

[7484650] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:19:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484650/

[7484652] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:19:37
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484652/

[7484653] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nvmeof 3-final}
time:   00:20:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484653/

[7484654] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:26
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484654/

[7484655] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_adoption}
time:   00:11:56
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484655/

[7484656] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:20:25
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484656/

[7484657] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:22:33
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484657/

[7484658] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
time:   00:44:40
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484658/

[7484659] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys}
time:   00:38:33
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484659/

[7484660] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:20:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484660/

[7484661] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_ca_signed_key}
time:   00:21:25
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484661/

[7484663] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:19:03
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484663/

[7484664] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:17:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484664/

[7484666] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_cephadm}
time:   00:17:49
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484666/

[7484667] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:59
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484667/

[7484668] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:19:47
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484668/

[7484669] orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/on orchestrator_cli}
time:   00:13:52
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484669/

[7484670] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:20:27
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484670/

[7484671] orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final}
time:   00:18:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484671/

[7484672] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:13:35
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484672/

[7484673] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
time:   00:50:19
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484673/

[7484675] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
time:   00:39:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484675/

[7484676] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_cephadm_repos}
time:   00:11:55
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484676/

[7484677] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final}
time:   00:18:57
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484677/

[7484678] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:03
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484678/

[7484681] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:24:19
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484681/

[7484682] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:19:34
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484682/

[7484683] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
time:   00:18:16
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484683/

[7484684] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_host_drain}
time:   00:23:46
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484684/

[7484685] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
time:   00:42:45
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484685/

[7484686] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
time:   00:28:19
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484686/

[7484687] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:21:59
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484687/

[7484688] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:19:39
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484688/

[7484689] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:19:03
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484689/

[7484691] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:17:34
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484691/

[7484692] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:21:09
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484692/

[7484693] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:14:11
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484693/

[7484694] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:22:30
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484694/

[7484696] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_monitoring_stack_basic}
time:   00:31:54
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484696/

[7484699] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:22:10
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484699/

[7484700] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:15:27
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484700/

[7484702] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:16:49
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484702/

[7484703] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:18:15
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484703/

[7484706] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:21:12
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484706/

[7484709] orch/cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs}
time:   00:31:14
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484709/

[7484711] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:36:26
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484711/

[7484712] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:15:02
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484712/

[7484713] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final}
time:   00:16:34
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484713/

[7484714] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:14:22
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484714/

[7484718] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:14:40
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484718/

[7484719] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:20:35
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484719/

[7484720] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nvmeof 3-final}
time:   00:16:22
info:   https://pulpito.ceph.com/yuriw-2023-12-08_19:38:02-orch-main-distro-default-smithi/7484720/