2022-11-21T16:36:40.391 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-21T16:36:40.392 INFO:teuthology.results:seed: '1207' 2022-11-21T16:37:23.183 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-21T16:37:23.184 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-11-21T17:07:25.112 INFO:teuthology.results:Tests finished! gathering results... 2022-11-21T17:07:25.415 INFO:scrape:Found 10 jobs 2022-11-21T17:07:25.551 INFO:scrape:Found 2 distinct failure reasons 2022-11-21T17:07:25.552 INFO:scrape:Failure: Command failed on smithi143 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a935e4a4-69bc-11ed-8438-001a4aab830c -e sha1=910cff70b59f29e584f743f9b020c9d5056f86a3 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' 2022-11-21T17:07:25.552 INFO:scrape:8 jobs: ['7086093', '7086099', '7086100', '7086095', '7086101', '7086096', '7086097', '7086098'] 2022-11-21T17:07:25.552 INFO:scrape:suites intersection: [] 2022-11-21T17:07:25.552 INFO:scrape:suites union: ['0-nvme-loop', '1-bootstrap/16.2.0', '1-start', '2-nfs', '2-ops/repave-all}', '2-repo_digest/defaut', '2-services/basic', '2-thrash', '3-final}', '3-tasks/rados_api_tests', '3-upgrade-with-workload', '3-upgrade/simple', '4-final}', '4-wait', '5-upgrade-ls', 'agent/off', 'base/install', 'cluster/{fixed-3', 'fixed-2', 'mon_election/classic', 'mon_election/classic}', 'msgr/async-v1only', 'openstack}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/rbd_iscsi/{0-single-container-host', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools}', 'orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'root}', 'start}', 'workloads/cephadm_iscsi}'] 2022-11-21T17:07:25.553 INFO:scrape: 2022-11-21T17:07:25.553 INFO:scrape:Failure: Command failed on smithi116 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' 2022-11-21T17:07:25.553 INFO:scrape:1 jobs: ['7086092'] 2022-11-21T17:07:25.553 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/pacific'] 2022-11-21T17:07:25.553 INFO:scrape: 2022-11-21T17:07:25.554 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/ failed: 9 dead: 0 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [7086092] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:11:09 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086092/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086092/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2def6cfe5b9d41cb86aa8338ed3f7dda Command failed on smithi116 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:pacific pull' [7086093] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:22:41 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086093/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086093/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c5a734442e584b71829f01d5d1bce71a Command failed on smithi143 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a935e4a4-69bc-11ed-8438-001a4aab830c -e sha1=910cff70b59f29e584f743f9b020c9d5056f86a3 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' [7086095] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:07:31 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086095/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086095/ sentry: https://sentry.ceph.com/organizations/ceph/?query=096f1c8e42d04e1c837a1383dce6bc3a Command failed on smithi088 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086096] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} ----------------------------------------------------------------- time: 00:07:31 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086096/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086096/ sentry: https://sentry.ceph.com/organizations/ceph/?query=170ec3d6f86847789475487488c8da54 Command failed on smithi055 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086097] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:07:38 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086097/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086097/ sentry: https://sentry.ceph.com/organizations/ceph/?query=89baa51e29a14732bc75ffbf4ef19339 Command failed on smithi121 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086098] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:07:35 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086098/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086098/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b82fcac891b44d92a1e6428e66077b9a Command failed on smithi111 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086099] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:07:23 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086099/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086099/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cd6f0dffef8047868fd51aa5e5878df8 Command failed on smithi167 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086100] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:10:30 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086100/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086100/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aefc2110d96047f899fe6d7111130257 Command failed on smithi099 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:910cff70b59f29e584f743f9b020c9d5056f86a3 pull' [7086101] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} ----------------------------------------------------------------- time: 00:18:33 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086101/ log: http://qa-proxy.ceph.com/teuthology/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086101/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cab46c7430e148198a86664bb44c611c Command failed on smithi047 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91982dd4-69bc-11ed-8438-001a4aab830c -e sha1=910cff70b59f29e584f743f9b020c9d5056f86a3 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' Pass ================================================================= [7086094] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools} 2-node-mgr agent/off orchestrator_cli} time: 00:16:14 info: http://pulpito.ceph.com/trociny-2022-11-21_16:36:33-orch:cephadm-wip-mgolub-testing-distro-default-smithi/7086094/