2022-01-04T05:02:16.925 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-04T05:02:16.928 INFO:teuthology.results:seed: '5837' 2022-01-04T05:27:41.333 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-04T05:27:41.361 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-04T07:00:48.220 INFO:teuthology.results:Tests finished! gathering results... 2022-01-04T07:00:49.901 INFO:scrape:Found 24 jobs 2022-01-04T07:00:50.338 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593585/teuthology.log 2022-01-04T07:00:51.843 INFO:scrape:Found 3 distinct failure reasons 2022-01-04T07:00:51.844 INFO:scrape:Failure: {'smithi105.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi060.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi017.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-01-04T07:00:51.845 INFO:scrape:5 jobs: ['6593576', '6593578', '6593574', '6593586', '6593575'] 2022-01-04T07:00:51.845 INFO:scrape:suites intersection: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install'] 2022-01-04T07:00:51.846 INFO:scrape:suites union: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install', 'test/kclient_workunit_suites_fsstress}}', 'test/kclient_workunit_suites_pjd}}', 'test/libcephfs_interface_tests}}', 'test/rados_api_tests}}', 'test/rbd_cli_import_export}}'] 2022-01-04T07:00:51.846 INFO:scrape: 2022-01-04T07:00:51.847 INFO:scrape:Dead: None 2022-01-04T07:00:51.847 INFO:scrape:2 jobs: ['6593585', '6593569'] 2022-01-04T07:00:51.847 INFO:scrape: 2022-01-04T07:00:51.848 INFO:scrape:Failure: Command failed on smithi183 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' 2022-01-04T07:00:51.848 INFO:scrape:1 jobs: ['6593572'] 2022-01-04T07:00:51.849 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{0-install', 'test/kclient_workunit_direct_io}}'] 2022-01-04T07:00:51.849 INFO:scrape: 2022-01-04T07:00:51.851 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/ failed: 1 dead: 7 running: 0 waiting: 0 queued: 0 passed: 16 Fail ================================================================= [6593572] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_direct_io}} ----------------------------------------------------------------- time: 00:10:36 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593572/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593572/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f339ed2f82f24cd7ad00afa52bbb63b9 Command failed on smithi183 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' Dead ================================================================= [6593569] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593569/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593569/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6593574] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:05:33 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593574/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593574/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e1606cf890f5434ba8598569d8310858 {'smithi163.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi059.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi049.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6593575] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:05:24 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593575/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593575/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02291c6e2c74483eae0de4226d5c7b48 {'smithi073.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi082.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi134.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6593576] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/libcephfs_interface_tests}} ----------------------------------------------------------------- time: 00:05:34 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593576/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593576/ sentry: https://sentry.ceph.com/organizations/ceph/?query=384c17cd32d7428ea447a4b3948515ed {'smithi105.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi060.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi017.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6593578] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_api_tests}} ----------------------------------------------------------------- time: 00:05:30 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593578/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593578/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36681bf132374a54a063163657472d52 {'smithi131.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi164.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi022.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6593585] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rbd_api_tests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593585/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593585/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6593586] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_cli_import_export}} ----------------------------------------------------------------- time: 00:05:24 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593586/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593586/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c77d84482cb474fadfaa6613bb04e4a {'smithi160.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi123.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi063.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} Pass ================================================================= [6593568] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/cfuse_workunit_suites_blogbench}} time: 00:18:19 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593568/ [6593570] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_iozone}} time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593570/ [6593571] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_pjd}} time: 00:12:34 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593571/ [6593573] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/kclient_workunit_suites_dbench}} time: 01:09:44 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593573/ [6593577] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/mon_thrash}} time: 00:30:26 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593577/ [6593579] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_bench}} time: 01:33:18 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593579/ [6593580] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rados_cache_snaps}} time: 00:22:54 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593580/ [6593581] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rados_cls_all}} time: 00:18:46 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593581/ [6593582] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rados_ec_snaps}} time: 00:24:55 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593582/ [6593583] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rados_python}} time: 00:17:45 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593583/ [6593584] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_workunit_loadgen_mix}} time: 00:21:56 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593584/ [6593587] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_fsx}} time: 00:13:24 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593587/ [6593588] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_python_api_tests}} time: 00:22:07 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593588/ [6593589] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:17:58 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593589/ [6593590] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/{0-install test/rgw_ec_s3tests}} time: 00:23:58 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593590/ [6593591] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rgw_s3tests}} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2022-01-04_05:00:03-smoke-master-distro-default-smithi/6593591/