2023-04-08T16:35:20.064 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-08T16:35:20.065 INFO:teuthology.results:subset: '111/12000' 2023-04-08T16:35:20.066 INFO:teuthology.results:seed: '8395' 2023-04-08T17:55:19.427 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-08T17:55:19.461 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-04-08T19:08:25.233 INFO:teuthology.results:Tests finished! gathering results... 2023-04-08T19:08:26.404 INFO:scrape:Found 15 jobs 2023-04-08T19:08:27.483 INFO:scrape:Found 3 distinct failure reasons 2023-04-08T19:08:27.484 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2023-04-08T19:08:27.485 INFO:scrape:1 jobs: ['7235666'] 2023-04-08T19:08:27.485 INFO:scrape:suites: ['ignorelist_health', 'objectstore/bluestore-comp-zstd', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'tasks/cfuse_workunit_suites_ffsb', 'thrashosds-health}'] 2023-04-08T19:08:27.486 INFO:scrape: 2023-04-08T19:08:27.486 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi142.front.sepia.ceph.com: ['type=AVC msg=audit(1680977012.862:45): avc: denied { node_bind } for pid=1623 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1680976462.691:44): avc: denied { node_bind } for pid=3839 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1680977249.866:45): avc: denied { node_bind } for pid=1613 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2023-04-08T19:08:27.486 INFO:scrape:4 jobs: ['7235678', '7235660', '7235672', '7235686'] 2023-04-08T19:08:27.487 INFO:scrape:suites intersection: ['ignorelist_health', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'thrashosds-health}'] 2023-04-08T19:08:27.487 INFO:scrape:suites union: ['ignorelist_health', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'tasks/admin_socket_objecter_requests', 'tasks/cfuse_workunit_suites_fsync', 'tasks/rados_api_tests', 'tasks/snaps-few-objects', 'thrashosds-health}'] 2023-04-08T19:08:27.487 INFO:scrape: 2023-04-08T19:08:27.488 INFO:scrape:Failure: Command failed on smithi023 with status 1: 'sudo yum install -y kernel' 2023-04-08T19:08:27.488 INFO:scrape:2 jobs: ['7235676', '7235670'] 2023-04-08T19:08:27.489 INFO:scrape:suites intersection: ['ignorelist_health', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_8', 'thrashosds-health}'] 2023-04-08T19:08:27.490 INFO:scrape:suites union: ['ignorelist_health', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_8', 'tasks/cfuse_workunit_suites_fsx', 'tasks/cfuse_workunit_suites_truncate_delay', 'thrashosds-health}'] 2023-04-08T19:08:27.490 INFO:scrape: 2023-04-08T19:08:27.492 INFO:teuthology.results:Sending results to yweinste@radhat.com: Test Run: yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/ failed: 7 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [7235660] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/rhel_8 tasks/admin_socket_objecter_requests thrashosds-health} ----------------------------------------------------------------- time: 00:17:45 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235660/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235660/ SELinux denials found on ubuntu@smithi202.front.sepia.ceph.com: ['type=AVC msg=audit(1680973869.170:44): avc: denied { node_bind } for pid=3228 comm="ping" saddr=172.21.15.202 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7235666] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_ffsb thrashosds-health} ----------------------------------------------------------------- time: 00:40:13 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235666/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235666/ reached maximum tries (50) after waiting for 300 seconds [7235670] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235670/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235670/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c0d131ab0df74220834b398f920a9521 Command failed on smithi119 with status 1: 'sudo yum install -y kernel' [7235672] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsync thrashosds-health} ----------------------------------------------------------------- time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235672/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235672/ SELinux denials found on ubuntu@smithi203.front.sepia.ceph.com: ['type=AVC msg=audit(1680975385.299:44): avc: denied { node_bind } for pid=3853 comm="ping" saddr=172.21.15.203 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7235676] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_truncate_delay thrashosds-health} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235676/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235676/ sentry: https://sentry.ceph.com/organizations/ceph/?query=32eedf1aa2ef4538ba90690a421f7426 Command failed on smithi023 with status 1: 'sudo yum install -y kernel' [7235678] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/rhel_8 tasks/rados_api_tests thrashosds-health} ----------------------------------------------------------------- time: 00:30:45 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235678/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235678/ SELinux denials found on ubuntu@smithi142.front.sepia.ceph.com: ['type=AVC msg=audit(1680977012.862:45): avc: denied { node_bind } for pid=1623 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1680976462.691:44): avc: denied { node_bind } for pid=3839 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1', 'type=AVC msg=audit(1680977249.866:45): avc: denied { node_bind } for pid=1613 comm="ping" saddr=172.21.15.142 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [7235686] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-all-distro/rhel_8 tasks/snaps-few-objects thrashosds-health} ----------------------------------------------------------------- time: 00:28:37 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235686/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235686/ SELinux denials found on ubuntu@smithi122.front.sepia.ceph.com: ['type=AVC msg=audit(1680977974.581:45): avc: denied { node_bind } for pid=3128 comm="ping" saddr=172.21.15.122 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] Pass ================================================================= [7235658] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/centos_8 tasks/snaps-many-objects thrashosds-health} time: 01:10:07 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235658/ [7235662] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} time: 01:05:38 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235662/ [7235664] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_misc thrashosds-health} time: 01:26:03 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235664/ [7235668] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health} time: 00:25:53 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235668/ [7235674] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_pjd thrashosds-health} time: 00:15:03 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235674/ [7235680] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/ubuntu_latest tasks/radosbench thrashosds-health} time: 01:01:52 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235680/ [7235683] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/readwrite thrashosds-health} time: 00:52:54 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235683/ [7235689] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-many-objects thrashosds-health} time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2023-04-08_15:50:19-powercycle-reef-distro-default-smithi/7235689/