2023-06-08T03:32:14.039 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-08T03:32:14.040 INFO:teuthology.results:seed: '8493'
2023-06-08T03:32:19.280 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-08T03:32:19.281 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-08T15:33:12.940 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-06-08T15:33:13.382 INFO:scrape:Found 6 jobs
2023-06-08T15:33:15.517 INFO:scrape:Found 4 distinct failure reasons
2023-06-08T15:33:15.519 INFO:scrape:Dead: b'2023-06-08T15:33:11.988 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf'
2023-06-08T15:33:15.519 INFO:scrape:ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)
 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0dace42520]
 2: pthread_kill()
 3: raise()
 4: abort()
 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x17b) [0x55fbd824d415]
 6: (KernelDevice::_aio_thread()+0x9bc) [0x55fbd8b0746c]
 7: ceph-osd(+0x157dd21) [0x55fbd8b07d21]
 8: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f0dace94b43]
 9: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f0dacf26a00]
2023-06-08T15:33:15.520 INFO:scrape:1 jobs: ['7298113']
2023-06-08T15:33:15.520 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/xfstests-dev}']
2023-06-08T15:33:15.521 INFO:scrape:
2023-06-08T15:33:15.522 INFO:scrape:Dead: b'2023-06-08T15:32:46.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf'
2023-06-08T15:33:15.522 INFO:scrape:ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)
 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f1b7ee12cf0]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1b80aa9e19]
 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f1b80aa9f85]
 6: (SimpleLock::put_xlock()+0x332) [0x562bd4e49432]
 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator<MutationImpl::LockOp> const&, MutationImpl*, bool*)+0x8c) [0x562bd4e3134c]
 8: (Locker::_drop_locks(MutationImpl*, std::set<CInode*, std::less<CInode*>, std::allocator<CInode*> >*, bool)+0x407) [0x562bd4e3a187]
 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set<CInode*, std::less<CInode*>, std::allocator<CInode*> >*)+0x5f) [0x562bd4e3e9ff]
 10: (Server::reply_client_request(boost::intrusive_ptr<MDRequestImpl>&, boost::intrusive_ptr<MClientReply> const&)+0x497) [0x562bd4c90bd7]
 11: (Server::respond_to_request(boost::intrusive_ptr<MDRequestImpl>&, int)+0x350) [0x562bd4c91a80]
 12: (Server::_unlink_local_finish(boost::intrusive_ptr<MDRequestImpl>&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x562bd4cafc3a]
 13: (MDSContext::complete(int)+0x5f) [0x562bd4f7818f]
 14: (MDSIOContextBase::complete(int)+0x524) [0x562bd4f78914]
 15: (MDSLogContextBase::complete(int)+0x5b) [0x562bd4f78ccb]
 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f1b80b4d7cd]
 17: /lib64/libpthread.so.0(+0x81ca) [0x7f1b7ee081ca]
 18: clone()
2023-06-08T15:33:15.523 INFO:scrape:3 jobs: ['7298110', '7298109', '7298112']
2023-06-08T15:33:15.523 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/xfstests-dev}']
2023-06-08T15:33:15.524 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/xfstests-dev}']
2023-06-08T15:33:15.525 INFO:scrape:
2023-06-08T15:33:15.525 INFO:scrape:Dead: b'2023-06-08T15:32:58.668 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf'
2023-06-08T15:33:15.526 INFO:scrape:ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)
 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f177fcb8ce0]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x197) [0x560fc6102dc3]
 5: (KernelDevice::_aio_thread()+0x1798) [0x560fc6d158d8]
 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560fc6d17f71]
 7: /lib64/libpthread.so.0(+0x81cf) [0x7f177fcae1cf]
 8: clone()
2023-06-08T15:33:15.527 INFO:scrape:1 jobs: ['7298111']
2023-06-08T15:33:15.528 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/xfstests-dev}']
2023-06-08T15:33:15.529 INFO:scrape:
2023-06-08T15:33:15.530 INFO:scrape:Failure: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev)
2023-06-08T15:33:15.531 INFO:scrape:1 jobs: ['7298108']
2023-06-08T15:33:15.532 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/xfstests-dev}']
2023-06-08T15:33:15.533 INFO:scrape: