2022-03-20T14:52:09.865 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-20T14:52:09.867 INFO:teuthology.results:subset: '111/120000' 2022-03-20T14:52:09.867 INFO:teuthology.results:seed: '4860' 2022-03-20T17:09:32.231 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-20T17:09:32.270 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-20T18:02:36.478 INFO:teuthology.results:Tests finished! gathering results... 2022-03-20T18:02:44.137 INFO:scrape:Found 131 jobs 2022-03-20T18:02:44.243 INFO:scrape:Found 1 distinct failure reasons 2022-03-20T18:02:44.244 INFO:scrape:Failure: Command failed on smithi138 with status 126: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:0613bec84365d24ea94584d17e6a6d0b292a650f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66abc616-a870-11ec-8c35-001a4aab830c -- ceph osd stat -f json' 2022-03-20T18:02:44.244 INFO:scrape:1 jobs: ['6749017'] 2022-03-20T18:02:44.245 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-services/nfs2', '3-final}', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8'] 2022-03-20T18:02:44.245 INFO:scrape: 2022-03-20T18:02:44.250 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 130 Fail ================================================================= [6749017] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749017/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749017/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d473c1c2abe14f80a95039f63cb28193 Command failed on smithi138 with status 126: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0613bec84365d24ea94584d17e6a6d0b292a650f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66abc616-a870-11ec-8c35-001a4aab830c -- ceph osd stat -f json' Pass ================================================================= [6748904] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:12:55 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748904/ [6748905] orch/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:46:15 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748905/ [6748906] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:25:31 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748906/ [6748907] orch/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:29:53 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748907/ [6748908] orch/cephadm/workunits/{agent/on mon_election/classic task/test_adoption} time: 00:07:26 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748908/ [6748909] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748909/ [6748910] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748910/ [6748911] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:14:45 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748911/ [6748912] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:12:55 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748912/ [6748913] orch/cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748913/ [6748914] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:29:31 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748914/ [6748915] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:15:49 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748915/ [6748916] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:40:06 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748916/ [6748917] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:37:29 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748917/ [6748918] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:44 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748918/ [6748919] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748919/ [6748920] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:30:53 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748920/ [6748921] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748921/ [6748922] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748922/ [6748923] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748923/ [6748924] orch/cephadm/workunits/{agent/on mon_election/classic task/test_cephadm_repos} time: 00:04:34 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748924/ [6748925] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 00:45:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748925/ [6748926] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:21:22 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748926/ [6748927] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:10 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748927/ [6748928] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:23 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748928/ [6748929] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:18 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748929/ [6748930] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:13:54 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748930/ [6748931] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/1.7.2} time: 00:17:50 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748931/ [6748932] orch/cephadm/workunits/{agent/off mon_election/connectivity task/test_nfs} time: 00:24:23 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748932/ [6748933] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748933/ [6748934] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:33:38 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748934/ [6748935] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli} time: 00:23:13 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748935/ [6748936] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:28:56 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748936/ [6748937] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748937/ [6748938] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:11:40 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748938/ [6748939] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:28:09 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748939/ [6748940] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:30:10 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748940/ [6748941] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:33:03 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748941/ [6748942] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748942/ [6748943] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:12:42 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748943/ [6748944] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:18:20 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748944/ [6748945] orch/cephadm/workunits/{agent/on mon_election/classic task/test_orch_cli} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748945/ [6748946] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748946/ [6748947] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748947/ [6748948] orch/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:45:56 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748948/ [6748949] orch/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:31:00 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748949/ [6748950] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748950/ [6748951] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:04 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748951/ [6748952] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748952/ [6748953] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:25:32 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748953/ [6748954] orch/cephadm/workunits/{agent/off mon_election/connectivity task/test_adoption} time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748954/ [6748955] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:16:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748955/ [6748956] orch/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748956/ [6748957] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:05 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748957/ [6748958] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:12:32 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748958/ [6748959] orch/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:41:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748959/ [6748960] orch/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:42:35 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748960/ [6748961] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748961/ [6748962] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748962/ [6748963] orch/cephadm/workunits/{agent/on mon_election/classic task/test_cephadm} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748963/ [6748964] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/master} time: 00:30:25 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748964/ [6748965] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:28 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748965/ [6748966] orch/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748966/ [6748967] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:17:20 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748967/ [6748968] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr agent/on orchestrator_cli} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748968/ [6748969] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748969/ [6748970] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:59:22 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748970/ [6748971] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:31:14 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748971/ [6748972] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:55 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748972/ [6748973] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:13:53 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748973/ [6748974] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:15:03 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748974/ [6748975] orch/cephadm/workunits/{agent/off mon_election/connectivity task/test_cephadm_repos} time: 00:04:31 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748975/ [6748976] orch/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748976/ [6748977] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:13:56 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748977/ [6748978] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:37 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748978/ [6748979] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:05 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748979/ [6748980] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:12:04 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748980/ [6748981] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:28:57 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748981/ [6748982] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:33:25 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748982/ [6748983] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748983/ [6748984] orch/cephadm/workunits/{agent/on mon_election/classic task/test_nfs} time: 00:28:24 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748984/ [6748985] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:12:03 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748985/ [6748986] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:30:35 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748986/ [6748987] orch/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:07 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748987/ [6748988] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:27 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748988/ [6748989] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748989/ [6748990] orch/cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:33:14 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748990/ [6748991] orch/cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:21:32 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748991/ [6748992] orch/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748992/ [6748993] orch/cephadm/workunits/{agent/off mon_election/connectivity task/test_orch_cli} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748993/ [6748994] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:14:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748994/ [6748995] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:12:26 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748995/ [6748996] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748996/ [6748997] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.2} time: 00:31:45 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748997/ [6748998] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:43 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748998/ [6748999] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:40 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6748999/ [6749000] orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} time: 00:10:51 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749000/ [6749001] orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:26:32 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749001/ [6749002] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:13:00 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749002/ [6749003] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:15:06 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749003/ [6749004] orch/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749004/ [6749005] orch/cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:42:06 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749005/ [6749006] orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:24:59 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749006/ [6749007] orch/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:43:59 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749007/ [6749008] orch/cephadm/workunits/{agent/off mon_election/classic task/test_adoption} time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749008/ [6749009] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749009/ [6749010] orch/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:11:40 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749010/ [6749011] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:14:25 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749011/ [6749012] orch/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749012/ [6749013] orch/cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749013/ [6749014] orch/cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 01:01:55 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749014/ [6749015] orch/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:30:33 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749015/ [6749016] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:59 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749016/ [6749018] orch/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:14:27 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749018/ [6749019] orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:30:09 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749019/ [6749020] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749020/ [6749021] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749021/ [6749022] orch/cephadm/workunits/{agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:26 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749022/ [6749023] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:25:05 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749023/ [6749024] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749024/ [6749025] orch/cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:37:24 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749025/ [6749026] orch/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:38:28 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749026/ [6749027] orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:30 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749027/ [6749028] orch/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749028/ [6749029] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749029/ [6749030] orch/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/master} time: 00:17:10 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749030/ [6749031] orch/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749031/ [6749032] orch/cephadm/workunits/{agent/on mon_election/connectivity task/test_nfs} time: 00:29:05 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749032/ [6749033] orch/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:14:19 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749033/ [6749034] orch/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr agent/on orchestrator_cli} time: 00:23:08 info: http://pulpito.ceph.com/yuriw-2022-03-20_14:30:27-orch-quincy-distro-default-smithi/6749034/