2021-11-11T16:20:11.317 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T16:20:11.319 INFO:teuthology.results:subset: '3/4' 2021-11-11T16:20:11.319 INFO:teuthology.results:seed: '7' 2021-11-11T16:33:39.112 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T16:33:39.116 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-12T04:34:32.732 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-11-12T04:34:33.199 INFO:scrape:Found 5 jobs 2021-11-12T04:34:34.396 INFO:scrape:Found 2 distinct failure reasons 2021-11-12T04:34:34.397 INFO:scrape:Failure: Command failed on smithi022 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' 2021-11-12T04:34:34.398 INFO:scrape:4 jobs: ['6497678', '6497681', '6497680', '6497682'] 2021-11-12T04:34:34.398 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'whitelist_wrongly_marked_down}'] 2021-11-12T04:34:34.399 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/multifs/{begin', 'fs/snaps/{begin', 'fs/thrash/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'overrides/{mon-debug', 'overrides/{no_client_pidfile', 'overrides/{whitelist_health', 'session_timeout', 'tasks/acls}', 'tasks/multifs-auth}', 'tasks/workunit/snaps}', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-12T04:34:34.399 INFO:scrape: 2021-11-12T04:34:34.400 INFO:scrape:Dead: b'2021-11-12T04:31:48.739 DEBUG:teuthology.exit:Finished running handlers' 2021-11-12T04:34:34.400 INFO:scrape:1 jobs: ['6497679'] 2021-11-12T04:34:34.401 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{osd-asserts', 'rhel_8}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-12T04:34:34.401 INFO:scrape: 2021-11-12T04:34:34.402 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/ failed: 4 dead: 1 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [6497678] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:12:33 info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497678/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497678/ sentry: https://sentry.ceph.com/organizations/ceph/?query=45d20737e8fa477e90dfa7a17960ccb1 Command failed on smithi022 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' [6497680] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:12:06 info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497680/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497680/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed1504157a704bf9ae45f9607694011f Command failed on smithi151 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' [6497681] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:11:02 info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497681/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497681/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e2e9e959553b42ba8ccad0cb65f4f815 Command failed on smithi131 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' [6497682] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:15:23 info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497682/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497682/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c698b5a89b434bee812520aed843ae74 Command failed on smithi117 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' Dead ================================================================= [6497679] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497679/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_16:19:33-fs-wip-kcephfs-new-mount-syntax-20211110-231624-testing-default-smithi/6497679/ hit max job timeout