2021-05-25T11:52:27.541 INFO:root:teuthology version: 1.1.0-d92ec140 2021-05-25T11:52:27.543 INFO:teuthology.results:seed: '4757' 2021-05-25T12:04:09.534 INFO:root:teuthology version: 1.1.0-d92ec140 2021-05-25T12:04:09.540 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-05-26T00:04:58.367 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-05-26T00:04:59.057 INFO:scrape:Found 8 jobs 2021-05-26T00:05:04.670 INFO:scrape:Found 3 distinct failure reasons 2021-05-26T00:05:04.671 INFO:scrape:Dead: b'2021-05-26T00:04:51.395 DEBUG:teuthology.orchestra.run.smithi168:> sudo logrotate /etc/logrotate.d/ceph-test.conf' 2021-05-26T00:05:04.671 INFO:scrape:ceph version 17.0.0-4201-g200922c0 (200922c02b33e9428d197c401fb1a5bec9001e32) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fb2866abb20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b6) [0x7fb287905d41] 5: (operator<<(std::ostream&, ClientMetricType const&)+0x10e) [0x7fb287bbc57e] 6: (MClientMetrics::print(std::ostream&) const+0x1b1) [0x7fb287bbc761] 7: (DispatchQueue::pre_dispatch(boost::intrusive_ptr const&)+0x710) [0x7fb287b6def0] 8: (DispatchQueue::fast_dispatch(boost::intrusive_ptr const&)+0x32) [0x7fb287b6fe62] 9: (ProtocolV1::handle_message_footer(char*, int)+0x1500) [0x7fb287c2c7e0] 10: /usr/lib64/ceph/libceph-common.so.2(+0x59ae0d) [0x7fb287c23e0d] 11: (AsyncConnection::process()+0x789) [0x7fb287c12269] 12: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1547) [0x7fb287c6eab7] 13: /usr/lib64/ceph/libceph-common.so.2(+0x5eca11) [0x7fb287c75a11] 14: /lib64/libstdc++.so.6(+0xc2ba3) [0x7fb285ae4ba3] 15: /lib64/libpthread.so.0(+0x814a) [0x7fb2866a114a] 16: clone() 2021-05-26T00:05:04.672 INFO:scrape:4 jobs: ['6134037', '6134032', '6134033', '6134036'] 2021-05-26T00:05:04.672 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client.yaml', 'conf/{client.yaml', 'fs:functional:/{begin.yaml', 'k-testing.yaml}', 'mds.yaml', 'mon.yaml', 'mount/kclient/{mount.yaml', 'ms-die-on-skipped.yaml}}', 'no_client_pidfile.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'tasks/mds-full.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2021-05-26T00:05:04.673 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client.yaml', 'conf/{client.yaml', 'distro/{centos_8.yaml}', 'distro/{rhel_8.yaml}', 'distro/{ubuntu_latest.yaml}', 'fs:functional:/{begin.yaml', 'k-testing.yaml}', 'mds.yaml', 'mon.yaml', 'mount/kclient/{mount.yaml', 'ms-die-on-skipped.yaml}}', 'no_client_pidfile.yaml', 'objectstore/bluestore-bitmap.yaml', 'objectstore/bluestore-ec-root.yaml', 'osd.yaml}', 'overrides/{distro/testing/{flavor/centos_latest.yaml', 'overrides/{distro/testing/{flavor/ubuntu_latest.yaml', 'overrides/{frag_enable.yaml', 'tasks/mds-full.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2021-05-26T00:05:04.673 INFO:scrape: 2021-05-26T00:05:04.674 INFO:scrape:Failure: Test failure: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2021-05-26T00:05:04.674 INFO:scrape:1 jobs: ['6134031'] 2021-05-26T00:05:04.675 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client.yaml', 'conf/{client.yaml', 'distro/{centos_8.yaml}', 'fs:functional:/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'no_client_pidfile.yaml', 'objectstore/bluestore-ec-root.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'tasks/mds-full.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2021-05-26T00:05:04.675 INFO:scrape: 2021-05-26T00:05:04.676 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2021-05-26T00:05:04.676 INFO:scrape:2 jobs: ['6134030', '6134034'] 2021-05-26T00:05:04.676 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client.yaml', 'conf/{client.yaml', 'fs:functional:/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/kclient/{mount.yaml', 'ms-die-on-skipped.yaml}}', 'no_client_pidfile.yaml', 'osd.yaml}', 'overrides/{distro/stock/{k-stock.yaml', 'overrides/{frag_enable.yaml', 'rhel_8.yaml}', 'tasks/mds-full.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2021-05-26T00:05:04.677 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client.yaml', 'conf/{client.yaml', 'distro/{rhel_8.yaml}', 'distro/{ubuntu_latest.yaml}', 'fs:functional:/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/kclient/{mount.yaml', 'ms-die-on-skipped.yaml}}', 'no_client_pidfile.yaml', 'objectstore/bluestore-bitmap.yaml', 'objectstore/bluestore-ec-root.yaml', 'osd.yaml}', 'overrides/{distro/stock/{k-stock.yaml', 'overrides/{frag_enable.yaml', 'rhel_8.yaml}', 'tasks/mds-full.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2021-05-26T00:05:04.677 INFO:scrape: