2023-12-15T14:24:41.121 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-15T14:24:41.121 INFO:teuthology.results:subset: '111/120000'
2023-12-15T14:24:41.122 INFO:teuthology.results:seed: '9'
2023-12-15T16:08:14.717 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-15T16:08:14.746 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-12-15T17:44:22.949 INFO:teuthology.results:Tests finished! gathering results...
2023-12-15T17:44:32.651 INFO:scrape:Found 141 jobs
2023-12-15T17:44:34.559 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492802/teuthology.log
2023-12-15T17:44:35.829 INFO:scrape:Found 8 distinct failure reasons
2023-12-15T17:44:35.830 INFO:scrape:Failure: Test failure: test_non_existent_cluster (tasks.cephfs.test_nfs.TestNFS)
2023-12-15T17:44:35.830 INFO:scrape:2 jobs: ['7492782', '7492851']
2023-12-15T17:44:35.830 INFO:scrape:suites intersection: ['task/test_nfs}']
2023-12-15T17:44:35.830 INFO:scrape:suites union: ['agent/off', 'agent/on', 'mon_election/classic', 'mon_election/connectivity', 'orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'task/test_nfs}']
2023-12-15T17:44:35.830 INFO:scrape:
2023-12-15T17:44:35.830 INFO:scrape:Failure: Command failed on smithi187 with status 5: 'sudo systemctl stop ceph-66fea984-9b68-11ee-95a5-87774f69a715@mon.smithi187'
2023-12-15T17:44:35.831 INFO:scrape:1 jobs: ['7492869']
2023-12-15T17:44:35.831 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/v16.2.4']
2023-12-15T17:44:35.831 INFO:scrape:
2023-12-15T17:44:35.831 INFO:scrape:Failure: SSH connection to smithi144 was lost: 'sudo apt-get update'
2023-12-15T17:44:35.831 INFO:scrape:1 jobs: ['7492803']
2023-12-15T17:44:35.831 INFO:scrape:suites: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', '3-final', 'cluster/1-node', 'k8s/1.21', 'net/host', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}']
2023-12-15T17:44:35.831 INFO:scrape:
2023-12-15T17:44:35.831 INFO:scrape:Failure: Command failed on smithi050 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3af8de9f7fd998cabb0ad31dc02733d37b33a399 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51f5f56e-9b5f-11ee-95a5-87774f69a715 -- ceph osd stat -f json'
2023-12-15T17:44:35.831 INFO:scrape:2 jobs: ['7492800', '7492829']
2023-12-15T17:44:35.831 INFO:scrape:suites intersection: []
2023-12-15T17:44:35.832 INFO:scrape:suites union: ['0-nvme-loop', '1-ranks/1', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-services/nfs-keepalive-only', '3-final}', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/v16.2.4']
2023-12-15T17:44:35.832 INFO:scrape:
2023-12-15T17:44:35.832 INFO:scrape:Failure: '2023-12-15T15:30:20.476422+00:00 smithi053 kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
' in syslog
2023-12-15T17:44:35.832 INFO:scrape:1 jobs: ['7492755']
2023-12-15T17:44:35.832 INFO:scrape:suites: ['1-bootstrap/16.2.5', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools']
2023-12-15T17:44:35.832 INFO:scrape:
2023-12-15T17:44:35.832 INFO:scrape:Failure: Command failed on smithi050 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"
2023-12-15T17:44:35.832 INFO:scrape:3 jobs: ['7492763', '7492838', '7492715']
2023-12-15T17:44:35.832 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '3-final', 'k8s/1.21', 'orch/rook/smoke/{0-distro/ubuntu_20.04']
2023-12-15T17:44:35.832 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}']
2023-12-15T17:44:35.832 INFO:scrape:
2023-12-15T17:44:35.833 INFO:scrape:Dead: None
2023-12-15T17:44:35.833 INFO:scrape:1 jobs: ['7492802']
2023-12-15T17:44:35.833 INFO:scrape:
2023-12-15T17:44:35.833 INFO:scrape:Failure: Command failed on smithi111 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0d1af04-9b5d-11ee-95a5-87774f69a715 -e sha1=3af8de9f7fd998cabb0ad31dc02733d37b33a399 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2023-12-15T17:44:35.833 INFO:scrape:1 jobs: ['7492770']
2023-12-15T17:44:35.833 INFO:scrape:suites: ['2-repo_digest/repo_digest', '3-upgrade/simple', '4-wait', '5-upgrade-ls', 'agent/off', 'mon_election/classic}', 'orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04']
2023-12-15T17:44:35.833 INFO:scrape:
2023-12-15T17:44:35.838 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/
failed:  10
dead:    2
running: 0
waiting: 0
queued:  0
passed:  129


Fail
=================================================================
[7492715]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:19
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492715/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492715/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cb5bcca6d0524403bc1fad4ca23a2254

    Command failed on smithi005 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7492755]  orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:33:07
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492755/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492755/

    '2023-12-15T15:30:20.476422+00:00 smithi053 kernel: rcu: INFO: rcu_sched
    detected stalls on CPUs/tasks: ' in syslog

[7492763]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/master}
-----------------------------------------------------------------
time:   00:06:24
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492763/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492763/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fdf9b55e863a487eb9bd6a587f8d0c5f

    Command failed on smithi050 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7492770]  orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
-----------------------------------------------------------------
time:   00:57:18
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492770/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492770/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eab37fecb27041898b49e093c84cf592

    Command failed on smithi111 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    a0d1af04-9b5d-11ee-95a5-87774f69a715 -e
    sha1=3af8de9f7fd998cabb0ad31dc02733d37b33a399 -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7492782]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:31:49
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492782/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492782/
sentry: https://sentry.ceph.com/organizations/ceph/?query=15bccf15771847ba82c2279384a02d89

    Test failure: test_non_existent_cluster (tasks.cephfs.test_nfs.TestNFS)

[7492800]  orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
-----------------------------------------------------------------
time:   00:15:08
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492800/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492800/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5f931a6ad0754bf89773662ec1853152

    Command failed on smithi050 with status 127: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:3af8de9f7fd998cabb0ad31dc02733d37b33a399 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    51f5f56e-9b5f-11ee-95a5-87774f69a715 -- ceph osd stat -f json'

[7492829]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:14:29
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492829/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492829/
sentry: https://sentry.ceph.com/organizations/ceph/?query=68b3ec89b3a04c83a5d48c8cd68eeaf1

    Command failed on smithi160 with status 127: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    3e1bb1f6-9b63-11ee-95a5-87774f69a715 -- ceph mon dump -f json'

[7492838]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master}
-----------------------------------------------------------------
time:   00:06:33
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492838/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492838/
sentry: https://sentry.ceph.com/organizations/ceph/?query=588fa45fb8fe46278153f8a66cbb601c

    Command failed on smithi116 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7492851]  orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:25:36
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492851/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492851/
sentry: https://sentry.ceph.com/organizations/ceph/?query=01f96a028dba4f9f97865e02433d94ac

    Test failure: test_non_existent_cluster (tasks.cephfs.test_nfs.TestNFS)

[7492869]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:35:45
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492869/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492869/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d0f67114b7f7478b9fa101a6efd3b8e5

    Command failed on smithi187 with status 5: 'sudo systemctl stop
    ceph-66fea984-9b68-11ee-95a5-87774f69a715@mon.smithi187'



Dead
=================================================================
[7492802]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492802/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492802/

    Error reimaging machines: Expected smithi144's OS to be centos 8 but found
    ubuntu 20.04

[7492803]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/1.7.2}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492803/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492803/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ecd386f8687544dbbbcdaf402790ec8f

    SSH connection to smithi144 was lost: 'sudo apt-get update'



Pass
=================================================================
[7492713] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:17:36
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492713/

[7492717] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon}
time:   00:25:34
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492717/

[7492719] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:19
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492719/

[7492721] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:35:07
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492721/

[7492723] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli}
time:   00:15:21
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492723/

[7492725] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:40:16
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492725/

[7492727] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:17
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492727/

[7492729] orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final}
time:   00:17:38
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492729/

[7492731] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
time:   00:38:56
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492731/

[7492734] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
time:   00:27:12
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492734/

[7492736] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:40:56
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492736/

[7492738] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:15:51
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492738/

[7492740] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:14:43
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492740/

[7492741] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite}
time:   00:18:06
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492741/

[7492742] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:19:26
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492742/

[7492743] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:16:09
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492743/

[7492744] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:15:25
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492744/

[7492745] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
time:   00:51:41
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492745/

[7492746] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:27:48
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492746/

[7492747] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:12
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492747/

[7492748] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations}
time:   00:23:39
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492748/

[7492749] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:15:30
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492749/

[7492750] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:16:23
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492750/

[7492751] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
time:   00:53:42
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492751/

[7492752] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:19:53
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492752/

[7492753] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption}
time:   00:16:29
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492753/

[7492754] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:17:32
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492754/

[7492756] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
time:   00:42:43
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492756/

[7492757] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys}
time:   00:38:00
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492757/

[7492758] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:15:03
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492758/

[7492759] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:24
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492759/

[7492760] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm}
time:   00:19:58
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492760/

[7492761] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:16:03
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492761/

[7492762] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:18:01
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492762/

[7492764] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:15:31
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492764/

[7492765] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:13:59
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492765/

[7492766] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos}
time:   00:09:34
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492766/

[7492767] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli}
time:   00:21:05
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492767/

[7492768] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final}
time:   00:15:57
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492768/

[7492769] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
time:   00:38:23
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492769/

[7492771] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
time:   00:36:28
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492771/

[7492772] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:14:24
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492772/

[7492773] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:14:34
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492773/

[7492774] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:25
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492774/

[7492775] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:16:33
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492775/

[7492776] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:15:58
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492776/

[7492777] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:24:16
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492777/

[7492778] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:15:29
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492778/

[7492779] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:34:52
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492779/

[7492780] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
time:   00:35:35
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492780/

[7492781] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
time:   00:25:52
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492781/

[7492783] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:20:30
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492783/

[7492785] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:18:01
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492785/

[7492788] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:37
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492788/

[7492790] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:15:43
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492790/

[7492792] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
time:   00:45:01
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492792/

[7492793] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:17:45
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492793/

[7492794] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli}
time:   00:20:50
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492794/

[7492795] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:18:00
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492795/

[7492796] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root}
time:   00:51:12
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492796/

[7492797] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys}
time:   00:33:05
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492797/

[7492798] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:13:50
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492798/

[7492799] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon}
time:   00:36:31
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492799/

[7492801] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:02
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492801/

[7492804] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:16:06
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492804/

[7492805] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite}
time:   00:16:36
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492805/

[7492806] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:35:42
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492806/

[7492807] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli}
time:   00:17:31
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492807/

[7492808] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:38:40
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492808/

[7492809] orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final}
time:   00:12:35
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492809/

[7492810] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
time:   00:34:40
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492810/

[7492811] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
time:   00:34:27
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492811/

[7492812] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:40:21
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492812/

[7492813] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:18:38
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492813/

[7492814] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:15:24
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492814/

[7492815] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:17:25
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492815/

[7492816] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:17:12
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492816/

[7492817] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations}
time:   00:18:46
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492817/

[7492818] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:29:44
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492818/

[7492819] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:16:50
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492819/

[7492820] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
time:   00:50:22
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492820/

[7492821] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:30:55
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492821/

[7492822] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption}
time:   00:17:16
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492822/

[7492823] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:13:26
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492823/

[7492824] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:17
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492824/

[7492825] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:15:49
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492825/

[7492826] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
time:   00:53:39
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492826/

[7492827] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:13:33
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492827/

[7492828] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm}
time:   00:20:31
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492828/

[7492830] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:18:53
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492830/

[7492831] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:31:53
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492831/

[7492832] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
time:   00:37:23
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492832/

[7492833] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys}
time:   00:27:36
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492833/

[7492834] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:25:12
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492834/

[7492835] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:02
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492835/

[7492836] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:16:07
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492836/

[7492837] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos}
time:   00:07:33
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492837/

[7492839] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:16:26
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492839/

[7492840] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:16:54
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492840/

[7492841] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:09
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492841/

[7492842] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/on orchestrator_cli}
time:   00:17:32
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492842/

[7492843] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:15:25
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492843/

[7492844] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final}
time:   00:15:45
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492844/

[7492845] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
time:   00:47:46
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492845/

[7492846] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic}
time:   00:28:56
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492846/

[7492847] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
time:   00:38:04
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492847/

[7492848] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:17:31
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492848/

[7492849] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:12:19
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492849/

[7492850] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:14:54
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492850/

[7492852] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:22:23
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492852/

[7492853] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:14:56
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492853/

[7492854] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final}
time:   00:35:29
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492854/

[7492855] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:38:41
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492855/

[7492856] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
time:   00:30:15
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492856/

[7492857] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:17:51
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492857/

[7492858] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   01:38:36
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492858/

[7492859] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli}
time:   00:21:16
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492859/

[7492860] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:18:13
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492860/

[7492861] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
time:   00:52:45
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492861/

[7492862] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:14:58
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492862/

[7492863] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli_mon}
time:   00:28:11
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492863/

[7492864] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:16:33
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492864/

[7492865] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:15:52
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492865/

[7492866] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:39:22
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492866/

[7492867] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys}
time:   00:29:16
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492867/

[7492868] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:20:56
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492868/

[7492870] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:17:14
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492870/

[7492871] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_rgw_multisite}
time:   00:18:52
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492871/

[7492872] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:20:27
info:   https://pulpito.ceph.com/yuriw-2023-12-14_22:28:35-orch-wip-yuri11-testing-2023-12-14-1108-quincy-distro-default-smithi/7492872/