Found 392 jobs Found 4 distinct failure reasons Failure: reached maximum tries (150) after waiting for 900 seconds 1 jobs: ['5183323'] suites: ['objectstore/bluestore-stupid', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_latest', 'tasks/cfuse_workunit_suites_ffsb', 'thrashosds-health', 'whitelist_health}'] Crash: Command failed (workunit test suites/fsx.sh) on smithi094 with status 161: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b340acf629a010a74d90da5782a2c5fe0b54ac20 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' ceph version 14.2.10 (b340acf629a010a74d90da5782a2c5fe0b54ac20) nautilus (stable) 1: (()+0x11390) [0x7f4e5ffaa390] 2: (gsignal()+0x38) [0x7f4e5f6f7428] 3: (abort()+0x16a) [0x7f4e5f6f902a] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x7f4e60485783] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f4e6048590d] 6: (ObjectCacher::Object::discard(long, long, C_GatherBuilderBase >*)+0x715) [0x529075] 7: (ObjectCacher::_discard(ObjectCacher::ObjectSet*, std::vector > const&, C_GatherBuilderBase >*)+0x166) [0x529246] 8: (ObjectCacher::discard_writeback(ObjectCacher::ObjectSet*, std::vector > const&, Context*)+0x6f) [0x53012f] 9: (Client::_invalidate_inode_cache(Inode*, long, long)+0xfd) [0x46e1bd] 10: (Client::_fallocate(Fh*, int, long, long)+0x275) [0x4cd095] 11: (Client::ll_fallocate(Fh*, int, long, long)+0x209) [0x4ce209] 12: ceph-fuse() [0x44ff9b] 13: (()+0x13a42) [0x7f4e6918aa42] 14: (()+0x15679) [0x7f4e6918c679] 15: (()+0x11e38) [0x7f4e69188e38] 16: (()+0x76ba) [0x7f4e5ffa06ba] 17: (clone()+0x6d) [0x7f4e5f7c941d] 2 jobs: ['5183437', '5183297'] suites intersection: ['powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/ubuntu_16.04', 'tasks/cfuse_workunit_suites_fsx', 'thrashosds-health', 'whitelist_health}'] suites union: ['objectstore/bluestore-comp-snappy', 'objectstore/filestore-xfs', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/ubuntu_16.04', 'tasks/cfuse_workunit_suites_fsx', 'thrashosds-health', 'whitelist_health}'] Failure: Timeout exceeded. command: /usr/bin/ipmitool args: ['/usr/bin/ipmitool', '-H', 'smithi049.ipmi.sepia.ceph.com', '-I', 'lanplus', '-U', 'inktank', '-P', 'ApGNXcA7', 'power', 'on'] buffer (last 100 chars): b'' after: match: None match_index: None exitstatus: None flag_eof: False pid: 25777 child_fd: 13 closed: False timeout: 30 delimiter: logfile: None logfile_read: None logfile_send: None maxread: 2000 ignorecase: False searchwindowsize: None delaybeforesend: 0.05 delayafterclose: 0.1 delayafterterminate: 0.1 searcher: searcher_re: 0: re.compile(b'Chassis Power Control: Up/On') 1: EOF 1 jobs: ['5183416'] suites: ['objectstore/bluestore-comp-lz4', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_7', 'tasks/snaps-few-objects', 'thrashosds-health', 'whitelist_health}'] Failure: Failed to revive osd.0 via ipmi 1 jobs: ['5183309'] suites: ['objectstore/filestore-xfs', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/ubuntu_16.04', 'tasks/cfuse_workunit_suites_ffsb', 'thrashosds-health', 'whitelist_health}']