2023-09-30T09:52:02.700 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-30T09:52:02.701 INFO:teuthology.results:subset: '111/120000'
2023-09-30T09:52:02.701 INFO:teuthology.results:seed: '3721'
2023-09-30T13:45:09.864 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-30T13:45:09.887 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-10-01T01:45:10.174 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-10-01T01:45:21.868 INFO:scrape:Found 155 jobs
2023-10-01T01:45:25.628 INFO:scrape:Found 5 distinct failure reasons
2023-10-01T01:45:25.629 INFO:scrape:Failure: Command failed on smithi134 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"
2023-10-01T01:45:25.629 INFO:scrape:4 jobs: ['7407342', '7407420', '7407477', '7407554']
2023-10-01T01:45:25.629 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'orch/rook/smoke/{0-distro/ubuntu_20.04']
2023-10-01T01:45:25.630 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}']
2023-10-01T01:45:25.630 INFO:scrape:
2023-10-01T01:45:25.630 INFO:scrape:Failure: No module named 'tasks.cephadm'
2023-10-01T01:45:25.630 INFO:scrape:4 jobs: ['7407454', '7407457', '7407451', '7407473']
2023-10-01T01:45:25.630 INFO:scrape:suites intersection: []
2023-10-01T01:45:25.630 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/nfs-ingress-rgw-user', '3-final}', 'agent/on', 'fixed-2', 'mode/packaged', 'mon_election/classic', 'msgr/async', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'start', 'task/test_iscsi_container/{centos_8.stream_container_tools', 'task/test_orch_cli_mon}', 'tasks/rados_python}', 'test_iscsi_container}}']
2023-10-01T01:45:25.630 INFO:scrape:
2023-10-01T01:45:25.631 INFO:scrape:Dead: b'2023-10-01T01:44:40.015 DEBUG:teuthology.exit:Finished running handlers'
2023-10-01T01:45:25.631 INFO:scrape:4 jobs: ['7407588', '7407386', '7407445', '7407465']
2023-10-01T01:45:25.631 INFO:scrape:suites intersection: []
2023-10-01T01:45:25.631 INFO:scrape:suites union: ['0-nvme-loop', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-services/nfs-ingress2', '3-final}', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/yes', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy']
2023-10-01T01:45:25.631 INFO:scrape:
2023-10-01T01:45:25.631 INFO:scrape:Failure: Command failed on smithi150 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ccdf176-5f7d-11ee-8db4-212e2dc638e7 -e sha1=1d10b71792f3be8887a7631e69851ac2df3585af -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2023-10-01T01:45:25.632 INFO:scrape:2 jobs: ['7407348', '7407481']
2023-10-01T01:45:25.632 INFO:scrape:suites intersection: ['1-bootstrap/17.2.0', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools']
2023-10-01T01:45:25.632 INFO:scrape:suites union: ['1-bootstrap/17.2.0', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools']
2023-10-01T01:45:25.632 INFO:scrape:
2023-10-01T01:45:25.632 INFO:scrape:Failure: reached maximum tries (301) after waiting for 300 seconds
2023-10-01T01:45:25.632 INFO:scrape:3 jobs: ['7407442', '7407544', '7407314']
2023-10-01T01:45:25.633 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}']
2023-10-01T01:45:25.633 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0']
2023-10-01T01:45:25.633 INFO:scrape:
2023-10-01T01:45:25.638 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/
failed:  13
dead:    3
running: 1
waiting: 0
queued:  0
passed:  138


Fail
=================================================================
[7407314]  orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/jaeger 3-final}
-----------------------------------------------------------------
time:   00:29:19
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407314/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407314/
sentry: https://sentry.ceph.com/organizations/ceph/?query=57fb4aa96817405db7b9221d08225e93

    reached maximum tries (301) after waiting for 300 seconds

[7407342]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:01
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407342/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407342/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5d841994008e4d4a8266700b82a82da2

    Command failed on smithi134 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7407348]  orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:31:29
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407348/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407348/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b7bae3a61ef4634a0b603f13005cb9b

    Command failed on smithi150 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    0ccdf176-5f7d-11ee-8db4-212e2dc638e7 -e
    sha1=1d10b71792f3be8887a7631e69851ac2df3585af -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7407420]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master}
-----------------------------------------------------------------
time:   00:06:10
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407420/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407420/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fb02dea3ae2d4db2bc34cb795d8ad803

    Command failed on smithi136 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7407442]  orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final}
-----------------------------------------------------------------
time:   00:31:38
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407442/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407442/
sentry: https://sentry.ceph.com/organizations/ceph/?query=11e53d3775c44fd1822bcbe07c928e20

    reached maximum tries (301) after waiting for 300 seconds

[7407451]  orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
-----------------------------------------------------------------
time:   00:17:24
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407451/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407451/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b762483df5be4c85b8730dedaff11984

    No module named 'tasks.workunit'

[7407454]  orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
-----------------------------------------------------------------
time:   00:11:35
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407454/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407454/
sentry: https://sentry.ceph.com/organizations/ceph/?query=26baec170d6c44f78efcca4d5f9ef75b

    No module named 'tasks.cephadm'

[7407457]  orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
-----------------------------------------------------------------
time:   00:15:06
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407457/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407457/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ae482dd3db2c4b8a95226ed20ea0a366

    No module named 'tasks.python'

[7407473]  orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:12:29
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407473/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407473/
sentry: https://sentry.ceph.com/organizations/ceph/?query=100c09fac7ff42c4a1f80de650412701

    No module named 'tasks'

[7407477]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2}
-----------------------------------------------------------------
time:   00:05:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407477/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407477/
sentry: https://sentry.ceph.com/organizations/ceph/?query=43f9200cb9ed47c98015894acfddf51e

    Command failed on smithi130 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"

[7407481]  orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final}
-----------------------------------------------------------------
time:   00:31:19
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407481/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407481/
sentry: https://sentry.ceph.com/organizations/ceph/?query=07a85a40ddcd4be69b4832634a042743

    Command failed on smithi150 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    93928c0e-5f8b-11ee-8db4-212e2dc638e7 -e
    sha1=1d10b71792f3be8887a7631e69851ac2df3585af -- bash -c \'ceph versions |
    jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

[7407544]  orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/jaeger 3-final}
-----------------------------------------------------------------
time:   00:36:36
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407544/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407544/
sentry: https://sentry.ceph.com/organizations/ceph/?query=132fa91823fe4960bef1b4d2d9b83332

    reached maximum tries (301) after waiting for 300 seconds

[7407554]  orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master}
-----------------------------------------------------------------
time:   00:06:13
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407554/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407554/
sentry: https://sentry.ceph.com/organizations/ceph/?query=98710fffe8c14de4844ab82a014650cd

    Command failed on smithi026 with status 100: "sudo apt update && sudo apt
    install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo
    /usr/share/keyrings/kubernetes-archive-keyring.gpg
    https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-
    by=/usr/share/keyrings/kubernetes-archive-keyring.gpg]
    https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee
    /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt
    install -y kubelet kubeadm kubectl bridge-utils"



Dead
=================================================================
[7407386]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407386/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407386/

    hit max job timeout

[7407445]  orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407445/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407445/

    hit max job timeout

[7407465]  orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407465/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407465/

    hit max job timeout



Running
=================================================================
[7407588] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407588/



Pass
=================================================================
[7407316] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:13:46
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407316/

[7407318] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:19:04
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407318/

[7407320] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:15:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407320/

[7407322] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:32:48
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407322/

[7407324] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root}
time:   00:36:50
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407324/

[7407326] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys}
time:   00:26:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407326/

[7407328] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:18:36
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407328/

[7407330] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:24:14
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407330/

[7407332] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
time:   00:19:55
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407332/

[7407334] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic}
time:   00:47:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407334/

[7407336] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:18:43
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407336/

[7407338] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:14:14
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407338/

[7407340] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_orch_cli}
time:   00:16:06
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407340/

[7407344] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:20:45
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407344/

[7407346] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:33:31
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407346/

[7407350] orch/cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{rhel_8} tasks/nfs}
time:   00:32:03
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407350/

[7407352] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/off orchestrator_cli}
time:   00:17:50
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407352/

[7407354] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:31:33
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407354/

[7407356] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:16:47
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407356/

[7407358] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:17:11
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407358/

[7407360] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final}
time:   00:15:22
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407360/

[7407362] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:14:14
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407362/

[7407364] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root}
time:   00:42:17
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407364/

[7407366] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests}
time:   00:43:33
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407366/

[7407368] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_orch_cli_mon}
time:   00:24:34
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407368/

[7407370] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final}
time:   00:16:41
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407370/

[7407372] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:14:07
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407372/

[7407374] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:19:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407374/

[7407376] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_rgw_multisite}
time:   00:21:23
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407376/

[7407378] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407378/

[7407380] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:17:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407380/

[7407382] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root}
time:   00:52:34
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407382/

[7407384] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python}
time:   00:26:40
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407384/

[7407388] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
time:   00:38:18
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407388/

[7407390] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_set_mon_crush_locations}
time:   00:25:22
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407390/

[7407392] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:18:28
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407392/

[7407394] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:20:37
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407394/

[7407396] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nvmeof 3-final}
time:   00:14:12
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407396/

[7407398] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:14:03
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407398/

[7407400] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_adoption}
time:   00:18:55
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407400/

[7407402] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:20:45
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407402/

[7407404] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:17:40
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407404/

[7407406] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root}
time:   00:41:54
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407406/

[7407408] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys}
time:   00:37:51
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407408/

[7407410] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:21:11
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407410/

[7407412] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:31:00
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407412/

[7407414] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_ca_signed_key}
time:   00:16:08
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407414/

[7407416] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:18:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407416/

[7407418] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:17:05
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407418/

[7407422] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:27
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407422/

[7407424] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:19:29
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407424/

[7407426] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_cephadm}
time:   00:18:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407426/

[7407428] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli}
time:   00:18:43
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407428/

[7407430] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:14:44
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407430/

[7407432] orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final}
time:   00:18:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407432/

[7407434] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:13:04
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407434/

[7407436] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root}
time:   00:37:30
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407436/

[7407438] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic}
time:   00:29:40
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407438/

[7407440] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
time:   00:37:22
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407440/

[7407443] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_cephadm_repos}
time:   00:09:42
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407443/

[7407444] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:43
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407444/

[7407446] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:24:22
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407446/

[7407447] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:19:41
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407447/

[7407448] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_extra_daemon_features}
time:   00:20:26
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407448/

[7407449] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
time:   00:18:18
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407449/

[7407450] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root}
time:   00:38:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407450/

[7407452] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:21:15
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407452/

[7407453] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:18:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407453/

[7407455] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:31:35
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407455/

[7407456] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:15:32
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407456/

[7407458] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:12:55
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407458/

[7407459] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:16:17
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407459/

[7407460] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity}
time:   01:01:17
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407460/

[7407461] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_orch_cli}
time:   00:18:41
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407461/

[7407462] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root}
time:   00:51:55
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407462/

[7407463] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys}
time:   00:31:43
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407463/

[7407467] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:15:10
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407467/

[7407469] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:12
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407469/

[7407471] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:19:20
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407471/

[7407475] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:30:56
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407475/

[7407479] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final}
time:   00:20:32
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407479/

[7407483] orch/cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs}
time:   00:36:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407483/

[7407485] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli}
time:   00:17:07
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407485/

[7407487] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi}
time:   00:34:54
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407487/

[7407489] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final}
time:   00:15:30
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407489/

[7407491] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final}
time:   00:16:02
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407491/

[7407493] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:13:45
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407493/

[7407495] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root}
time:   00:36:13
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407495/

[7407497] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests}
time:   00:41:20
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407497/

[7407499] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_rgw_multisite}
time:   00:17:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407499/

[7407501] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:15:37
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407501/

[7407503] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:20:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407503/

[7407505] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nvmeof 3-final}
time:   00:13:50
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407505/

[7407507] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_set_mon_crush_locations}
time:   00:24:33
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407507/

[7407509] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final}
time:   00:22:16
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407509/

[7407511] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:30:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407511/

[7407513] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic}
time:   00:30:12
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407513/

[7407515] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final}
time:   00:19:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407515/

[7407517] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root}
time:   00:49:27
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407517/

[7407519] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python}
time:   00:32:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407519/

[7407521] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_adoption}
time:   00:17:02
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407521/

[7407523] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:14:59
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407523/

[7407524] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:18:24
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407524/

[7407526] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final}
time:   00:18:38
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407526/

[7407528] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final}
time:   00:15:30
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407528/

[7407530] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:12:48
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407530/

[7407532] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_ca_signed_key}
time:   00:18:52
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407532/

[7407534] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final}
time:   00:15:03
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407534/

[7407536] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:29:57
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407536/

[7407538] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root}
time:   00:34:49
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407538/

[7407540] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys}
time:   00:25:32
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407540/

[7407542] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:25:11
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407542/

[7407546] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:18:01
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407546/

[7407548] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_cephadm}
time:   00:18:07
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407548/

[7407550] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final}
time:   00:17:26
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407550/

[7407552] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity}
time:   00:58:14
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407552/

[7407556] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final}
time:   00:20:31
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407556/

[7407558] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_cephadm_repos}
time:   00:10:03
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407558/

[7407560] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:33:24
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407560/

[7407562] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli}
time:   00:18:14
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407562/

[7407564] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:16:13
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407564/

[7407566] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final}
time:   00:15:20
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407566/

[7407568] orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final}
time:   00:19:15
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407568/

[7407570] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:13:38
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407570/

[7407572] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root}
time:   00:46:50
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407572/

[7407574] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests}
time:   00:37:48
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407574/

[7407576] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:16:21
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407576/

[7407578] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_extra_daemon_features}
time:   00:16:04
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407578/

[7407580] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
time:   00:15:37
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407580/

[7407582] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final}
time:   00:21:12
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407582/

[7407584] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:17:42
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407584/

[7407586] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:16:20
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407586/

[7407590] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root}
time:   00:39:24
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407590/

[7407592] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python}
time:   00:27:55
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407592/

[7407594] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:18:36
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407594/

[7407596] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:31:15
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407596/

[7407598] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final}
time:   00:20:24
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407598/

[7407600] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_orch_cli}
time:   00:19:36
info:   http://pulpito.ceph.com/yuriw-2023-09-29_19:44:36-orch-main-distro-default-smithi/7407600/