2024-02-15T17:43:05.409 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-15T17:43:05.409 INFO:teuthology.results:subset: '111/120000' 2024-02-15T17:43:05.409 INFO:teuthology.results:seed: '964' 2024-02-15T18:09:30.759 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-15T18:09:30.786 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-15T18:58:34.450 INFO:teuthology.results:Tests finished! gathering results... 2024-02-15T18:58:35.275 INFO:scrape:Found 27 jobs 2024-02-15T18:58:40.409 INFO:scrape:Found 5 distinct failure reasons 2024-02-15T18:58:40.410 INFO:scrape:Failure: "2024-02-15T18:18:09.443860+0000 mon.a (mon.0) 165 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] is down (out of quorum)" in cluster log 2024-02-15T18:58:40.410 INFO:scrape:7 jobs: ['7561667', '7561653', '7561654', '7561666', '7561656', '7561652', '7561671'] 2024-02-15T18:58:40.410 INFO:scrape:suites intersection: [] 2024-02-15T18:58:40.410 INFO:scrape:suites union: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'distro/ubuntu_18.04', 'fixed-2', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/classic', 'mon_election/connectivity', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'roles', 'start}', 'task/test_nfs}', 'task/test_orch_cli}', 'tasks/{0-from/v16.2.4'] 2024-02-15T18:58:40.410 INFO:scrape: 2024-02-15T18:58:40.410 INFO:scrape:Failure: "2024-02-15T18:29:18.653809+0000 mon.smithi012 (mon.0) 7 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data" in cluster log 2024-02-15T18:58:40.410 INFO:scrape:10 jobs: ['7561662', '7561664', '7561677', '7561660', '7561665', '7561655', '7561672', '7561675', '7561657', '7561668'] 2024-02-15T18:58:40.411 INFO:scrape:suites intersection: [] 2024-02-15T18:58:40.411 INFO:scrape:suites union: ['1-bootstrap/16.2.5', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-repo_digest/repo_digest', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '3-upgrade/staggered', '4-final}', '4-verify}', '4-verify}}', '4-wait', '5-upgrade-ls', 'centos_8.stream_container_tools', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon_election/connectivity}', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'roles', 'tasks/{0-from/v16.2.4'] 2024-02-15T18:58:40.411 INFO:scrape: 2024-02-15T18:58:40.411 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:58:40.411 INFO:scrape:2 jobs: ['7561674', '7561659'] 2024-02-15T18:58:40.411 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2024-02-15T18:58:40.411 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2024-02-15T18:58:40.411 INFO:scrape: 2024-02-15T18:58:40.411 INFO:scrape:Failure: expected string or bytes-like object 2024-02-15T18:58:40.411 INFO:scrape:1 jobs: ['7561669'] 2024-02-15T18:58:40.412 INFO:scrape:suites: ['1-start', '2-thrash', '3-tasks/rados_api_tests', 'fixed-2', 'msgr/async', 'rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'root}'] 2024-02-15T18:58:40.412 INFO:scrape: 2024-02-15T18:58:40.412 INFO:scrape:Failure: Command failed on smithi018 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 240ddaae-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2024-02-15T18:58:40.412 INFO:scrape:2 jobs: ['7561663', '7561678'] 2024-02-15T18:58:40.412 INFO:scrape:suites intersection: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2024-02-15T18:58:40.412 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2024-02-15T18:58:40.412 INFO:scrape: