2023-05-25T01:54:07.970 INFO:root:teuthology version: 0.0.1.dev109+ge5a3f9b 2023-05-25T01:54:07.972 INFO:teuthology.results:seed: '3204' 2023-05-25T01:56:45.890 INFO:root:teuthology version: 0.0.1.dev109+ge5a3f9b 2023-05-25T01:56:45.895 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-25T13:56:52.806 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-05-25T13:56:53.180 INFO:scrape:Found 4 jobs 2023-05-25T13:56:53.792 INFO:scrape:Found 2 distinct failure reasons 2023-05-25T13:56:53.793 INFO:scrape:Dead: b'Exception: Cannot connect to remote host smithi016' 2023-05-25T13:56:53.793 INFO:scrape:ceph version 18.0.0-3871-g5a6b166b (5a6b166b7c43e2b5890833e75dd257ab21264ded) reef (dev) 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f325de12cf0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f325f2a9e19] 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f325f2a9f85] 6: (Client::add_update_cap(Inode*, MetaSession*, unsigned long, unsigned int, unsigned int, unsigned int, unsigned int, inodeno_t, int, UserPerm const&)+0xc53) [0x55dfe8d99863] 7: (Client::add_update_inode(InodeStat*, utime_t, MetaSession*, UserPerm const&)+0x943) [0x55dfe8d9ac33] 8: (Client::insert_readdir_results(MetaRequest*, MetaSession*, Inode*, Inode*)+0x789) [0x55dfe8d9bd49] 9: (Client::insert_trace(MetaRequest*, MetaSession*)+0xfab) [0x55dfe8d9e2eb] 10: (Client::handle_client_reply(boost::intrusive_ptr const&)+0x3a0) [0x55dfe8d9ee70] 11: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x58d) [0x55dfe8dc182d] 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f325f522d88] 13: (DispatchQueue::entry()+0x50f) [0x7f325f51ff2f] 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f325f5e6221] 15: /lib64/libpthread.so.0(+0x81ca) [0x7f325de081ca] 16: clone() 2023-05-25T13:56:53.793 INFO:scrape:1 jobs: ['7286096'] 2023-05-25T13:56:53.794 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_8}', 'fs:mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2023-05-25T13:56:53.794 INFO:scrape: 2023-05-25T13:56:53.794 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi114 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=65708f43f4f9537b3e8888e6c188ea9be9e52cb1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' 2023-05-25T13:56:53.795 INFO:scrape:1 jobs: ['7286095'] 2023-05-25T13:56:53.795 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs:mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2023-05-25T13:56:53.795 INFO:scrape: 2023-05-25T13:56:53.796 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/ failed: 1 dead: 0 running: 1 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [7286095] fs:mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:25:32 info: http://pulpito.ceph.com/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7286095/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7286095/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b3040e3a1e546059544ef452839d98e Command failed (workunit test kernel_untar_build.sh) on smithi114 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=65708f43f4f9537b3e8888e6c188ea9be9e52cb1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' Running ================================================================= [7286096] fs:mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} info: http://pulpito.ceph.com/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7286096/ Pass ================================================================= [7286093] fs:mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:37:54 info: http://pulpito.ceph.com/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7286093/ [7286094] fs:mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:38:30 info: http://pulpito.ceph.com/mchangir-2023-05-25_01:52:06-fs:mixed-clients-wip-vshankar-testing-20230509.090020-1-distro-default-smithi/7286094/