2024-03-04T20:53:31.541 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-04T20:53:31.541 INFO:teuthology.results:subset: '111/120000' 2024-03-04T20:53:31.541 INFO:teuthology.results:seed: '9349' 2024-03-04T20:54:13.636 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-04T20:54:13.644 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-04T23:53:27.697 INFO:teuthology.results:Tests finished! gathering results... 2024-03-04T23:53:28.379 INFO:scrape:Found 10 jobs 2024-03-04T23:53:28.431 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581424/teuthology.log 2024-03-04T23:53:28.466 INFO:scrape:Found 3 distinct failure reasons 2024-03-04T23:53:28.467 INFO:scrape:Failure: ['Data could not be sent to remote host "smithi153.front.sepia.ceph.com". Make sure this host can be reached over ssh: ssh: connect to host smithi153.front.sepia.ceph.com port 22: No route to host'] 2024-03-04T23:53:28.467 INFO:scrape:1 jobs: ['7581426'] 2024-03-04T23:53:28.467 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd_api', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8'] 2024-03-04T23:53:28.467 INFO:scrape: 2024-03-04T23:53:28.467 INFO:scrape:Dead: None 2024-03-04T23:53:28.467 INFO:scrape:1 jobs: ['7581424'] 2024-03-04T23:53:28.467 INFO:scrape: 2024-03-04T23:53:28.468 INFO:scrape:Failure: Stale jobs detected, aborting. 2024-03-04T23:53:28.468 INFO:scrape:1 jobs: ['7581425'] 2024-03-04T23:53:28.468 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd-import-export', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0'] 2024-03-04T23:53:28.468 INFO:scrape: 2024-03-04T23:53:28.469 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/ failed: 2 dead: 1 running: 0 waiting: 0 queued: 0 passed: 7 Fail ================================================================= [7581425] upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581425/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581425/ sentry: https://sentry.ceph.com/organizations/ceph/?query=faca1ae0dd4547eebb77a5fb000cb874 Stale jobs detected, aborting. [7581426] upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 02:26:45 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581426/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581426/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aba59fbe93a942d6b28e109e275b29b5 ['Data could not be sent to remote host "smithi153.front.sepia.ceph.com". Make sure this host can be reached over ssh: ssh: connect to host smithi153.front.sepia.ceph.com port 22: No route to host'] Dead ================================================================= [7581424] upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581424/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581424/ Error reimaging machines: reached maximum tries (241) after waiting for 3600 seconds Pass ================================================================= [7581420] upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:21:28 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581420/ [7581421] upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:40:35 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581421/ [7581422] upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:39:55 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581422/ [7581423] upgrade:quincy-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:38:02 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581423/ [7581427] upgrade:quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_20.04} time: 00:09:03 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581427/ [7581428] upgrade:quincy-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:49:16 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581428/ [7581429] upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:28:12 info: https://pulpito.ceph.com/yuriw-2024-03-04_20:53:23-upgrade:quincy-x-reef-release-distro-default-smithi/7581429/