2023-04-30T23:46:00.943 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-04-30T23:46:00.944 INFO:teuthology.results:subset: '111/12000' 2023-04-30T23:46:00.944 INFO:teuthology.results:seed: '2568' 2023-05-01T00:07:03.134 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-01T00:07:03.161 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-01T01:27:09.562 INFO:teuthology.results:Tests finished! gathering results... 2023-05-01T01:27:10.528 INFO:scrape:Found 15 jobs 2023-05-01T01:27:11.938 INFO:scrape:Found 2 distinct failure reasons 2023-05-01T01:27:11.939 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi047.front.sepia.ceph.com: ['type=AVC msg=audit(1682899849.090:45): avc: denied { node_bind } for pid=1666 comm="ping" saddr=172.21.15.47 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2023-05-01T01:27:11.939 INFO:scrape:5 jobs: ['7257376', '7257385', '7257379', '7257388', '7257382'] 2023-05-01T01:27:11.940 INFO:scrape:suites intersection: ['ignorelist_health', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'thrashosds-health}'] 2023-05-01T01:27:11.940 INFO:scrape:suites union: ['ignorelist_health', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'tasks/admin_socket_objecter_requests', 'tasks/cfuse_workunit_suites_ffsb', 'tasks/cfuse_workunit_suites_fsync', 'tasks/rados_api_tests', 'tasks/snaps-few-objects', 'thrashosds-health}'] 2023-05-01T01:27:11.940 INFO:scrape: 2023-05-01T01:27:11.941 INFO:scrape:Failure: Command failed on smithi167 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 2023-05-01T01:27:11.941 INFO:scrape:5 jobs: ['7257378', '7257381', '7257384', '7257387', '7257375'] 2023-05-01T01:27:11.941 INFO:scrape:suites intersection: ['ignorelist_health', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_8', 'thrashosds-health}'] 2023-05-01T01:27:11.942 INFO:scrape:suites union: ['ignorelist_health', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_8', 'tasks/cfuse_workunit_misc', 'tasks/cfuse_workunit_suites_fsx', 'tasks/cfuse_workunit_suites_truncate_delay', 'tasks/readwrite', 'tasks/snaps-many-objects', 'thrashosds-health}'] 2023-05-01T01:27:11.942 INFO:scrape: 2023-05-01T01:27:11.943 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/ failed: 10 dead: 0 running: 0 waiting: 0 queued: 0 passed: 5 Fail ================================================================= [7257375] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/centos_8 tasks/snaps-many-objects thrashosds-health} ----------------------------------------------------------------- time: 00:07:09 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257375/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257375/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e24cc2ca8c3453aa465d24346b02e31 Command failed on smithi072 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel' [7257376] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/rhel_8 tasks/admin_socket_objecter_requests thrashosds-health} ----------------------------------------------------------------- time: 00:20:18 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257376/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257376/ SELinux denials found on ubuntu@smithi047.front.sepia.ceph.com: ['type=AVC msg=audit(1682899849.090:45): avc: denied { node_bind } for pid=1666 comm="ping" saddr=172.21.15.47 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7257378] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_misc thrashosds-health} ----------------------------------------------------------------- time: 00:07:11 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257378/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257378/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9df132c9ffe64f54890049db6470e412 Command failed on smithi167 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel' [7257379] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_ffsb thrashosds-health} ----------------------------------------------------------------- time: 00:32:13 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257379/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257379/ SELinux denials found on ubuntu@smithi081.front.sepia.ceph.com: ['type=AVC msg=audit(1682900340.260:44): avc: denied { node_bind } for pid=3844 comm="ping" saddr=172.21.15.81 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7257381] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health} ----------------------------------------------------------------- time: 00:07:33 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257381/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257381/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6bed7de9b5b48fdb435262eac6a3375 Command failed on smithi033 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel' [7257382] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsync thrashosds-health} ----------------------------------------------------------------- time: 00:17:55 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257382/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257382/ SELinux denials found on ubuntu@smithi112.front.sepia.ceph.com: ['type=AVC msg=audit(1682900312.378:44): avc: denied { node_bind } for pid=3204 comm="ping" saddr=172.21.15.112 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7257384] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_truncate_delay thrashosds-health} ----------------------------------------------------------------- time: 00:13:33 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257384/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257384/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d5020071213f46c9b655f8b334184ac4 Command failed on smithi137 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel' [7257385] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/rhel_8 tasks/rados_api_tests thrashosds-health} ----------------------------------------------------------------- time: 00:35:47 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257385/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257385/ SELinux denials found on ubuntu@smithi072.front.sepia.ceph.com: ['type=AVC msg=audit(1682901689.353:45): avc: denied { node_bind } for pid=1531 comm="ping" saddr=172.21.15.72 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7257387] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/readwrite thrashosds-health} ----------------------------------------------------------------- time: 00:07:16 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257387/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257387/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5c7956223c74e68b326ba80aa4da6e7 Command failed on smithi044 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils- libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs- devel' [7257388] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-all-distro/rhel_8 tasks/snaps-few-objects thrashosds-health} ----------------------------------------------------------------- time: 00:42:18 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257388/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257388/ SELinux denials found on ubuntu@smithi123.front.sepia.ceph.com: ['type=AVC msg=audit(1682902268.742:46): avc: denied { node_bind } for pid=1628 comm="ping" saddr=172.21.15.123 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] Pass ================================================================= [7257377] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} time: 01:25:55 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257377/ [7257380] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health} time: 00:28:41 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257380/ [7257383] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_pjd thrashosds-health} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257383/ [7257386] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/ubuntu_latest tasks/radosbench thrashosds-health} time: 01:07:13 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257386/ [7257389] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-many-objects thrashosds-health} time: 00:30:59 info: http://pulpito.ceph.com/yuriw-2023-04-28_19:00:39-powercycle-reef-distro-default-smithi/7257389/