2023-05-01T18:50:14.374 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-01T18:50:14.375 INFO:teuthology.results:subset: '1/999' 2023-05-01T18:50:14.376 INFO:teuthology.results:seed: '5759' 2023-05-01T19:34:02.128 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-01T19:34:02.163 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-01T21:24:10.616 INFO:teuthology.results:Tests finished! gathering results... 2023-05-01T21:24:12.251 INFO:scrape:Found 24 jobs 2023-05-01T21:24:12.672 INFO:scrape:Found 3 distinct failure reasons 2023-05-01T21:24:12.673 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=033dda5187874d8c9ac976b9867fd9bec436d557 2023-05-01T21:24:12.673 INFO:scrape:1 jobs: ['7259545'] 2023-05-01T21:24:12.673 INFO:scrape:suites: ['orch:cephadm/dashboard/{0-distro/ignorelist_health', 'task/test_e2e}'] 2023-05-01T21:24:12.674 INFO:scrape: 2023-05-01T21:24:12.674 INFO:scrape:Failure: Cannot connect to remote host smithi115 2023-05-01T21:24:12.674 INFO:scrape:2 jobs: ['7259553', '7259543'] 2023-05-01T21:24:12.675 INFO:scrape:suites intersection: [] 2023-05-01T21:24:12.675 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/nfs-ingress-rgw-bucket', '3-final}', 'fixed-2', 'mode/packaged', 'mon_election/classic', 'msgr/async', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'orch:cephadm/with-work/{0-distro/ubuntu_18.04', 'start', 'tasks/rados_api_tests}'] 2023-05-01T21:24:12.675 INFO:scrape: 2023-05-01T21:24:12.675 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=033dda5187874d8c9ac976b9867fd9bec436d557 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-05-01T21:24:12.676 INFO:scrape:1 jobs: ['7259536'] 2023-05-01T21:24:12.676 INFO:scrape:suites: ['orch:cephadm/dashboard/{0-distro/centos_8.stream_container_tools', 'task/test_e2e}'] 2023-05-01T21:24:12.676 INFO:scrape: 2023-05-01T21:24:12.678 INFO:teuthology.results:Sending results to adking@redhat.com: Test Run: adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 20 Fail ================================================================= [7259536] orch:cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} ----------------------------------------------------------------- time: 00:30:21 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259536/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259536/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52fa7542618e4a6684da31e79efd6dc6 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=033dda5187874d8c9ac976b9867fd9bec436d557 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7259543] orch:cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259543/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259543/ sentry: https://sentry.ceph.com/organizations/ceph/?query=90582450cdcc46dbb33f87e44ea767f3 Cannot connect to remote host smithi193 [7259545] orch:cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} ----------------------------------------------------------------- time: 00:07:33 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259545/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259545/ sentry: https://sentry.ceph.com/organizations/ceph/?query=87d0476edc3a43268978ab76d51affce Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1 =033dda5187874d8c9ac976b9867fd9bec436d557 [7259553] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259553/ log: http://qa-proxy.ceph.com/teuthology/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259553/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f3d16170caa14585bb8b80e29da6cf14 Cannot connect to remote host smithi115 Pass ================================================================= [7259532] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:37:56 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259532/ [7259533] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} time: 00:32:46 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259533/ [7259534] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:17:33 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259534/ [7259535] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:39:16 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259535/ [7259537] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:53:28 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259537/ [7259538] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:48:54 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259538/ [7259539] orch:cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:21:54 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259539/ [7259540] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:47:23 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259540/ [7259541] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:28:31 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259541/ [7259542] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:52:38 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259542/ [7259544] orch:cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} time: 00:25:15 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259544/ [7259546] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench} time: 01:38:34 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259546/ [7259547] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:37:21 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259547/ [7259548] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:59:27 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259548/ [7259549] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:28:37 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259549/ [7259550] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:46:49 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259550/ [7259551] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:51:53 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259551/ [7259552] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:37:58 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259552/ [7259554] orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:54:11 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259554/ [7259555] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} time: 00:36:24 info: http://pulpito.ceph.com/adking-2023-05-01_11:51:02-orch:cephadm-wip-adk4-testing-2023-04-25-1313-pacific-distro-default-smithi/7259555/