2023-07-23T15:17:35.882 INFO:root:teuthology version: 0.0.1.dev158+g1426592 2023-07-23T15:17:35.883 INFO:teuthology.results:subset: '111/120000' 2023-07-23T15:17:35.883 INFO:teuthology.results:seed: '5355' 2023-07-23T20:45:27.252 INFO:root:teuthology version: 0.0.1.dev158+g1426592 2023-07-23T20:45:27.286 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-07-23T21:32:31.508 INFO:teuthology.results:Tests finished! gathering results... 2023-07-23T21:32:41.610 INFO:scrape:Found 145 jobs 2023-07-23T21:32:44.423 INFO:scrape:Found 3 distinct failure reasons 2023-07-23T21:32:44.424 INFO:scrape:Failure: Command failed on smithi008 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9593fc2-299b-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" 2023-07-23T21:32:44.425 INFO:scrape:9 jobs: ['7348688', '7348551', '7348384', '7348625', '7348284', '7348342', '7348361', '7348420', '7348492'] 2023-07-23T21:32:44.425 INFO:scrape:suites intersection: [] 2023-07-23T21:32:44.425 INFO:scrape:suites union: ['1-bootstrap/16.2.0', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-nfs', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/yes', '3-upgrade-mgr-staggered', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-config-upgrade/{fail_fs}', '4-final}', '4-verify}', '4-wait', '5-upgrade-ls', '5-upgrade-with-workload', '6-verify}}', 'agent/off', 'agent/on', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/classic}', 'mon_election/connectivity}', 'orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/pacific'] 2023-07-23T21:32:44.426 INFO:scrape: 2023-07-23T21:32:44.426 INFO:scrape:Failure: reached maximum tries (301) after waiting for 300 seconds 2023-07-23T21:32:44.426 INFO:scrape:2 jobs: ['7348530', '7348354'] 2023-07-23T21:32:44.427 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}'] 2023-07-23T21:32:44.427 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0'] 2023-07-23T21:32:44.427 INFO:scrape: 2023-07-23T21:32:44.428 INFO:scrape:Failure: Command failed on smithi019 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 2023-07-23T21:32:44.428 INFO:scrape:4 jobs: ['7348348', '7348566', '7348671', '7348456'] 2023-07-23T21:32:44.428 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'orch/rook/smoke/{0-distro/ubuntu_20.04'] 2023-07-23T21:32:44.428 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] 2023-07-23T21:32:44.429 INFO:scrape: 2023-07-23T21:32:44.435 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/ failed: 15 dead: 0 running: 0 waiting: 0 queued: 0 passed: 130 Fail ================================================================= [7348284] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:27:10 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348284/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348284/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae9457dfc30c4f6e9f07ddb261fefd39 Command failed on smithi027 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 790614a2-296f-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348342] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:56:06 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348342/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348342/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a3265ddb73544b95b0a1f511c17c1da5 Command failed on smithi039 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 86938930-2977-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348348] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} ----------------------------------------------------------------- time: 00:06:18 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348348/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348348/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1892b5a3ec5c4c3ca76dd4a09ee765b1 Command failed on smithi019 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7348354] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/jaeger 3-final} ----------------------------------------------------------------- time: 00:37:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348354/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348354/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b6f74a6316a49429ac2090fb04d831e reached maximum tries (301) after waiting for 300 seconds [7348361] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348361/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348361/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1e3cd706b04549239ac30c4fbcff4dd5 Command failed on smithi087 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35b2edd2-297a-11ee-9b34-001a4aab830c -- ceph mon dump -f json' [7348384] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:18:49 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348384/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348384/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6357d75f47bb4b4abf3de449271b02fe Command failed on smithi175 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid aa738a20-297d-11ee-9b34-001a4aab830c -- ceph mon dump -f json' [7348420] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:44:28 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348420/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348420/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a43abfb99b141728f823aaccca6e752 Command failed on smithi078 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6fe5566-2980-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348456] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:06:05 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348456/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348456/ sentry: https://sentry.ceph.com/organizations/ceph/?query=caa27535fc7e4f1cb576eec176ee0e93 Command failed on smithi190 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7348492] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} ----------------------------------------------------------------- time: 00:34:31 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348492/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348492/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c06c64e7c554f4fa17e0eaa6293aa2d Command failed on smithi016 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c782c4dc-2987-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348530] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} ----------------------------------------------------------------- time: 00:27:26 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348530/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348530/ sentry: https://sentry.ceph.com/organizations/ceph/?query=72425cbfa03c489590e26bc47c3b1b57 reached maximum tries (301) after waiting for 300 seconds [7348551] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} ----------------------------------------------------------------- time: 00:44:30 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348551/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348551/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5e3268563371486b92ba292dc80ea09a Command failed on smithi081 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d5595c6e-298d-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348566] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:07 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348566/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348566/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b07143a0aa54bf190b8e53d1015809d Command failed on smithi173 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7348625] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:35:00 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348625/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348625/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b768c0ca09384646a9f95f91e602662f Command failed on smithi086 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e5998fe-2995-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" [7348671] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:12 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348671/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348671/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2d9a5f3d9684d3da775da0ee9663077 Command failed on smithi057 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7348688] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:27:29 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348688/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348688/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f81d46d8d67540b59a0a60c80609ae8e Command failed on smithi008 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a9593fc2-299b-11ee-9b34-001a4aab830c -- bash -c 'ceph orch upgrade ls'" Pass ================================================================= [7348280] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:15:01 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348280/ [7348282] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348282/ [7348286] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348286/ [7348288] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:20:20 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348288/ [7348290] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348290/ [7348292] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:38:09 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348292/ [7348294] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:48:03 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348294/ [7348297] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:32:02 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348297/ [7348300] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:20:57 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348300/ [7348303] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:33:23 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348303/ [7348306] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:18:18 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348306/ [7348309] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:18:31 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348309/ [7348312] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348312/ [7348315] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:17:05 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348315/ [7348318] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348318/ [7348321] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:19:09 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348321/ [7348324] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:58 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348324/ [7348327] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:55:27 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348327/ [7348330] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348330/ [7348333] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:14:53 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348333/ [7348336] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_nfs} time: 00:36:21 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348336/ [7348339] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348339/ [7348345] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:18:12 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348345/ [7348351] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348351/ [7348357] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:20:59 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348357/ [7348363] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli} time: 00:24:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348363/ [7348366] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:43:48 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348366/ [7348369] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:16:11 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348369/ [7348372] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348372/ [7348375] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:36:58 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348375/ [7348378] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:43:47 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348378/ [7348381] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:20:06 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348381/ [7348387] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:21:27 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348387/ [7348391] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:28:45 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348391/ [7348393] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348393/ [7348396] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:47 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348396/ [7348399] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348399/ [7348402] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:47:13 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348402/ [7348405] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:35:25 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348405/ [7348408] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} time: 00:25:56 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348408/ [7348411] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:16:10 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348411/ [7348414] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:15:25 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348414/ [7348417] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348417/ [7348423] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348423/ [7348426] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:19:05 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348426/ [7348429] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:28:41 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348429/ [7348432] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} time: 00:18:13 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348432/ [7348435] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:39:19 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348435/ [7348438] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:46:04 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348438/ [7348441] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:30:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348441/ [7348444] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:19:49 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348444/ [7348447] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:10:38 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348447/ [7348450] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:01 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348450/ [7348453] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348453/ [7348459] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:32:11 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348459/ [7348462] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:20 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348462/ [7348465] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348465/ [7348468] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} time: 00:18:00 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348468/ [7348471] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli} time: 00:22:43 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348471/ [7348474] orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:19:21 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348474/ [7348477] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:12:42 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348477/ [7348480] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:06:53 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348480/ [7348483] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:44:16 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348483/ [7348486] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:20:30 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348486/ [7348489] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:21:09 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348489/ [7348495] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:08:57 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348495/ [7348498] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348498/ [7348501] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:16:30 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348501/ [7348504] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348504/ [7348507] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:37:54 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348507/ [7348510] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:34:16 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348510/ [7348513] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:26:39 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348513/ [7348516] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348516/ [7348519] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:48 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348519/ [7348522] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:19 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348522/ [7348525] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:12:56 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348525/ [7348528] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:00 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348528/ [7348532] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:38:13 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348532/ [7348536] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:36:39 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348536/ [7348539] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:18:43 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348539/ [7348542] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:45:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348542/ [7348545] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:27:17 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348545/ [7348548] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:19 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348548/ [7348554] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:24:28 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348554/ [7348557] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:34 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348557/ [7348560] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348560/ [7348563] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:21:43 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348563/ [7348568] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:28:47 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348568/ [7348572] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:37:35 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348572/ [7348575] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:40:15 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348575/ [7348577] orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348577/ [7348580] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:37:58 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348580/ [7348583] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:17 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348583/ [7348586] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348586/ [7348589] orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348589/ [7348592] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:12:49 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348592/ [7348595] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:46:33 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348595/ [7348598] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:42:15 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348598/ [7348601] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:16:20 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348601/ [7348604] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348604/ [7348607] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:21:46 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348607/ [7348610] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} time: 00:23:06 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348610/ [7348613] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:16 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348613/ [7348616] orch/cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:53:32 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348616/ [7348619] orch/cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:29:48 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348619/ [7348622] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:18:15 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348622/ [7348628] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:33:10 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348628/ [7348631] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348631/ [7348634] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348634/ [7348637] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348637/ [7348640] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348640/ [7348643] orch/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:17:49 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348643/ [7348646] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:55 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348646/ [7348649] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:40:04 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348649/ [7348652] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:43:45 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348652/ [7348655] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:38:45 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348655/ [7348658] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:16:24 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348658/ [7348661] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:32:42 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348661/ [7348664] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:19:20 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348664/ [7348666] orch/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:57 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348666/ [7348669] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:18:32 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348669/ [7348673] orch/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348673/ [7348675] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348675/ [7348677] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:09:11 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348677/ [7348679] orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/on orchestrator_cli} time: 00:17:09 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348679/ [7348681] orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:13:37 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348681/ [7348682] orch/cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:12:45 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348682/ [7348685] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:36:53 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348685/ [7348691] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:41:05 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348691/ [7348694] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2023-07-23_14:01:49-orch-reef-distro-default-smithi/7348694/