2021-11-11T13:44:52.199 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T13:44:52.202 INFO:teuthology.results:subset: '3/4' 2021-11-11T13:44:52.202 INFO:teuthology.results:seed: '1285' 2021-11-11T13:46:57.614 INFO:root:teuthology version: 1.1.0-91815aeb 2021-11-11T13:46:57.620 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-12T01:47:51.518 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-11-12T01:47:52.048 INFO:scrape:Found 5 jobs 2021-11-12T01:47:53.492 INFO:scrape:Found 3 distinct failure reasons 2021-11-12T01:47:53.493 INFO:scrape:Failure: must be str, not NoneType 2021-11-12T01:47:53.493 INFO:scrape:3 jobs: ['6497502', '6497501', '6497499'] 2021-11-12T01:47:53.494 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'whitelist_wrongly_marked_down}'] 2021-11-12T01:47:53.494 INFO:scrape:suites union: ['clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/multifs/{begin', 'fs/snaps/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{mon-debug', 'overrides/{no_client_pidfile', 'overrides/{whitelist_health', 'tasks/acls}', 'tasks/multifs-auth}', 'tasks/workunit/snaps}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-12T01:47:53.495 INFO:scrape: 2021-11-12T01:47:53.495 INFO:scrape:Dead: b'socket.timeout' 2021-11-12T01:47:53.496 INFO:scrape:1 jobs: ['6497500'] 2021-11-12T01:47:53.496 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{osd-asserts', 'rhel_8}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-12T01:47:53.497 INFO:scrape: 2021-11-12T01:47:53.497 INFO:scrape:Failure: Command failed on smithi157 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' 2021-11-12T01:47:53.498 INFO:scrape:1 jobs: ['6497503'] 2021-11-12T01:47:53.498 INFO:scrape:suites: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-12T01:47:53.499 INFO:scrape: 2021-11-12T01:47:53.500 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/ failed: 4 dead: 0 running: 1 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [6497499] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:12:20 info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497499/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497499/ sentry: https://sentry.ceph.com/organizations/ceph/?query=80e040cce7254177be8f59fb1d4e7b74 must be str, not NoneType [6497501] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:12:24 info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497501/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497501/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ba3c7ed4ec442b28803132cd2c82356 must be str, not NoneType [6497502] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:13:55 info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497502/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497502/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9aba2fe20d7e45f6898a5c5e74837c48 must be str, not NoneType [6497503] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:12:37 info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497503/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497503/ sentry: https://sentry.ceph.com/organizations/ceph/?query=17df4a69288c418d9ba9ba096a01a519 Command failed on smithi157 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes -f' Running ================================================================= [6497500] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} info: http://pulpito.ceph.com/vshankar-2021-11-11_13:42:39-fs-wip-kcephfs-new-mount-syntax-20211110-231624-distro-default-smithi/6497500/