2023-02-06T22:01:52.147 INFO:root:teuthology version: 0.0.1.dev37+g00690ec
2023-02-06T22:01:52.148 INFO:teuthology.results:subset: '111/120000'
2023-02-06T22:01:52.148 INFO:teuthology.results:seed: '9723'
2023-02-06T22:29:48.709 INFO:root:teuthology version: 0.0.1.dev37+g00690ec
2023-02-06T22:29:48.714 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-02-07T01:16:01.136 INFO:teuthology.results:Tests finished! gathering results...
2023-02-07T01:16:04.469 INFO:scrape:Found 38 jobs
2023-02-07T01:16:09.434 INFO:scrape:Found 6 distinct failure reasons
2023-02-07T01:16:09.435 INFO:scrape:Failure: Command failed on smithi023 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull'
2023-02-07T01:16:09.435 INFO:scrape:11 jobs: ['7151347', '7151353', '7151352', '7151336', '7151346', '7151360', '7151355', '7151345', '7151358', '7151325', '7151351']
2023-02-07T01:16:09.436 INFO:scrape:suites intersection: []
2023-02-07T01:16:09.436 INFO:scrape:suites union: ['0-kubeadm', '1-ranks/1', '1-ranks/2', '1-rook', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-workload/none', '2-workload/radosbench', '3-final', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'cluster/1-node', 'cluster/3-node', 'conf/{client', 'ignorelist_wrongly_marked_down', 'k8s/1.21', 'mds', 'mon', 'net/calico', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/rook/smoke/{0-distro/ubuntu_18.04', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'roles', 'rook/1.6.2}', 'rook/master}', 'tasks/{0-from/v16.2.4']
2023-02-07T01:16:09.437 INFO:scrape:
2023-02-07T01:16:09.437 INFO:scrape:Failure: Command failed on smithi142 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cf4fe54c-a66e-11ed-bc20-001a4aab830c -e sha1=7d3a7c08b7ca6bb31d289b3ba424eaf8a3abad39 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''
2023-02-07T01:16:09.438 INFO:scrape:2 jobs: ['7151338', '7151333']
2023-02-07T01:16:09.438 INFO:scrape:suites intersection: ['1-ranks/2', '1-volume/{0-create', '2-client', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4']
2023-02-07T01:16:09.438 INFO:scrape:suites union: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4']
2023-02-07T01:16:09.439 INFO:scrape:
2023-02-07T01:16:09.439 INFO:scrape:Failure: Command failed on smithi007 with status 2: 'git archive --remote=https://git.ceph.com/ceph-ci.git 7d3a7c08b7ca6bb31d289b3ba424eaf8a3abad39 src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm'
2023-02-07T01:16:09.440 INFO:scrape:11 jobs: ['7151343', '7151356', '7151359', '7151327', '7151332', '7151324', '7151330', '7151340', '7151349', '7151348', '7151339']
2023-02-07T01:16:09.440 INFO:scrape:suites intersection: []
2023-02-07T01:16:09.440 INFO:scrape:suites union: ['0-nvme-loop', '1-install/nautilus', '1-install/nautilus-v1only', '1-install/nautilus-v2only', '1-install/octopus', '1-start', '2-ops/rm-zap-add}', '2-ops/rmdir-reactivate}', '2-services/rgw', '2-services/rgw-ingress', '3-final}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04}', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}']
2023-02-07T01:16:09.441 INFO:scrape:
2023-02-07T01:16:09.441 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d3a7c08b7ca6bb31d289b3ba424eaf8a3abad39 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2023-02-07T01:16:09.442 INFO:scrape:4 jobs: ['7151350', '7151331', '7151357', '7151341']
2023-02-07T01:16:09.442 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2023-02-07T01:16:09.443 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}']
2023-02-07T01:16:09.443 INFO:scrape:
2023-02-07T01:16:09.444 INFO:scrape:Failure: timeout expired in wait_until_healthy
2023-02-07T01:16:09.444 INFO:scrape:1 jobs: ['7151344']
2023-02-07T01:16:09.445 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4']
2023-02-07T01:16:09.445 INFO:scrape:
2023-02-07T01:16:09.446 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7d3a7c08b7ca6bb31d289b3ba424eaf8a3abad39
2023-02-07T01:16:09.446 INFO:scrape:2 jobs: ['7151354', '7151337']
2023-02-07T01:16:09.447 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}']
2023-02-07T01:16:09.447 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}']
2023-02-07T01:16:09.447 INFO:scrape: