2023-12-02T08:14:31.922 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-02T08:14:31.922 INFO:teuthology.results:subset: '111/120000'
2023-12-02T08:14:31.923 INFO:teuthology.results:seed: '4654'
2023-12-02T08:23:28.201 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-02T08:23:28.204 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-12-02T08:43:29.810 INFO:teuthology.results:Tests finished! gathering results...
2023-12-02T08:43:30.263 INFO:scrape:Found 9 jobs
2023-12-02T08:43:30.643 INFO:scrape:Found 1 distinct failure reasons
2023-12-02T08:43:30.643 INFO:scrape:Failure: Command failed on smithi158 with status 1: 'test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm'
2023-12-02T08:43:30.644 INFO:scrape:9 jobs: ['7475342', '7475347', '7475344', '7475339', '7475341', '7475346', '7475345', '7475340', '7475343']
2023-12-02T08:43:30.644 INFO:scrape:suites intersection: []
2023-12-02T08:43:30.644 INFO:scrape:suites union: ['0-roles', '0-start', '1-start', '1-tasks', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'snaps-few-objects}', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:reef-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'upgrade:reef-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8}', 'upgrade:reef-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:reef-x/stress-split/{0-distro/centos_8.stream_container_tools_crun', 'upgrade:reef-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0', 'upgrade:reef-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_20.04', 'workload/{ec-rados-default']
2023-12-02T08:43:30.644 INFO:scrape:
2023-12-02T08:43:30.645 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/
failed:  9
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7475339]  upgrade:reef-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity}
-----------------------------------------------------------------
time:   00:15:07
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475339/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475339/
sentry: https://sentry.ceph.com/organizations/ceph/?query=788992cd3a43452c88a02cb0b0a6c572

    Command failed on smithi088 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475340]  upgrade:reef-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic}
-----------------------------------------------------------------
time:   00:08:55
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475340/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475340/
sentry: https://sentry.ceph.com/organizations/ceph/?query=624c751e43114c85ad2c5081b7799932

    Command failed on smithi012 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475341]  upgrade:reef-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic}
-----------------------------------------------------------------
time:   00:11:25
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475341/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475341/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b0b22e10772c4457a20885f160f8e38b

    Command failed on smithi076 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475342]  upgrade:reef-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   00:11:46
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475342/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475342/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e4a6922022414bd89968ad720ddde7f9

    Command failed on smithi158 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475343]  upgrade:reef-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity}
-----------------------------------------------------------------
time:   00:12:58
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475343/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475343/
sentry: https://sentry.ceph.com/organizations/ceph/?query=835e122cf2c34610bf10d33b187eecc9

    Command failed on smithi031 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475344]  upgrade:reef-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic}
-----------------------------------------------------------------
time:   00:15:26
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475344/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475344/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6df2c37ebfac48f2b472202d8d4d8418

    Command failed on smithi006 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475345]  upgrade:reef-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity}
-----------------------------------------------------------------
time:   00:14:49
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475345/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475345/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f0d8530498d2420197ab93ec553cd963

    Command failed on smithi106 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475346]  upgrade:reef-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   00:13:33
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475346/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475346/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dac85e3a6003448d887a78e0e39eee7c

    Command failed on smithi032 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'

[7475347]  upgrade:reef-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic}
-----------------------------------------------------------------
time:   00:08:12
info:   https://pulpito.ceph.com/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475347/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-01_18:09:09-upgrade:reef-x-main-distro-default-smithi/7475347/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2c37476ae7d6414997d025dbd821ef07

    Command failed on smithi135 with status 1: 'test -s
    /home/ubuntu/cephtest/cephadm && test $(stat -c%s
    /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x
    /home/ubuntu/cephtest/cephadm'