2023-06-01T06:48:38.012 INFO:root:teuthology version: 0.0.1.dev116+g62814ab 2023-06-01T06:48:38.013 INFO:teuthology.results:seed: '2581' 2023-06-01T06:48:42.631 INFO:root:teuthology version: 0.0.1.dev116+g62814ab 2023-06-01T06:48:42.632 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-01T08:26:50.585 INFO:teuthology.results:Tests finished! gathering results... 2023-06-01T08:26:51.110 INFO:scrape:Found 8 jobs 2023-06-01T08:26:51.112 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293571/teuthology.log 2023-06-01T08:26:54.495 INFO:scrape:Found 4 distinct failure reasons 2023-06-01T08:26:54.496 INFO:scrape:Dead: None 2023-06-01T08:26:54.497 INFO:scrape:1 jobs: ['7293571'] 2023-06-01T08:26:54.497 INFO:scrape: 2023-06-01T08:26:54.497 INFO:scrape:Failure: Command failed on smithi105 with status 1: 'set -ex\nsudo dd of=/home/ubuntu/cephtest/valgrind.supp' 2023-06-01T08:26:54.498 INFO:scrape:2 jobs: ['7293570', '7293566'] 2023-06-01T08:26:54.498 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2023-06-01T08:26:54.498 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2023-06-01T08:26:54.499 INFO:scrape: 2023-06-01T08:26:54.499 INFO:scrape:Failure: {'smithi121.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['sudo', 'apt-get', 'clean'], 'delta': '0:00:00.013527', 'end': '2023-06-01 07:10:08.135287', 'invocation': {'module_args': {'_raw_params': 'sudo apt-get clean', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 100, 'start': '2023-06-01 07:10:08.121760', 'stderr': 'E: Could not get lock /var/cache/apt/archives/lock. It is held by process 6069 (apt-get)\nE: Unable to lock directory /var/cache/apt/archives/', 'stderr_lines': ['E: Could not get lock /var/cache/apt/archives/lock. It is held by process 6069 (apt-get)', 'E: Unable to lock directory /var/cache/apt/archives/'], 'stdout': '', 'stdout_lines': [], 'warnings': ["Consider using 'become', 'become_method', and 'become_user' rather than running sudo"]}} 2023-06-01T08:26:54.499 INFO:scrape:1 jobs: ['7293568'] 2023-06-01T08:26:54.500 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2023-06-01T08:26:54.500 INFO:scrape: 2023-06-01T08:26:54.501 INFO:scrape:Failure: Test failure: test_session_ls_with_filters (tasks.cephfs.test_misc.TestMisc) 2023-06-01T08:26:54.502 INFO:scrape:4 jobs: ['7293564', '7293569', '7293567', '7293565'] 2023-06-01T08:26:54.502 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2023-06-01T08:26:54.502 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2023-06-01T08:26:54.503 INFO:scrape: 2023-06-01T08:26:54.504 INFO:teuthology.results:Sending results to neesingh@redhat.com: Test Run: neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/ failed: 7 dead: 1 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [7293564] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 01:03:16 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293564/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293564/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e5c3041a94a4954a0fb29f731a2b216 Test failure: test_session_ls_with_filters (tasks.cephfs.test_misc.TestMisc) [7293565] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 01:04:55 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293565/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293565/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c093f819affa44f3bb8a265ba61f183d Test failure: test_session_ls_with_filters (tasks.cephfs.test_misc.TestMisc) [7293566] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:12:30 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293566/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293566/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df933e14a9534d1e9cdfcafb332bb314 Command failed on smithi026 with status 1: 'set -ex\nsudo dd of=/home/ubuntu/cephtest/valgrind.supp' [7293567] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 01:02:03 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293567/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293567/ sentry: https://sentry.ceph.com/organizations/ceph/?query=101b41c9405145258d14ec07f370cbbd Test failure: test_session_ls_with_filters (tasks.cephfs.test_misc.TestMisc) [7293568] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:10:56 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293568/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293568/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7babd74633a9491e9f8cfa53ba34afa5 {'smithi121.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['sudo', 'apt-get', 'clean'], 'delta': '0:00:00.013527', 'end': '2023-06-01 07:10:08.135287', 'invocation': {'module_args': {'_raw_params': 'sudo apt-get clean', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 100, 'start': '2023-06-01 07:10:08.121760', 'stderr': 'E: Could not get lock /var/cache/apt/archives/lock. It is held by process 6069 (apt-get)\nE: Unable to lock directory /var/cache/apt/archives/', 'stderr_lines': ['E: Could not get lock /var/cache/apt/archives/lock. It is held by process 6069 (apt-get)', 'E: Unable to lock directory /var/cache/apt/archives/'], 'stdout': '', 'stdout_lines': [], 'warnings': ["Consider using 'become', 'become_method', and 'become_user' rather than running sudo"]}} [7293569] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 01:02:30 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293569/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293569/ sentry: https://sentry.ceph.com/organizations/ceph/?query=25027be9a8c7447f8ae4239442f9d65a Test failure: test_session_ls_with_filters (tasks.cephfs.test_misc.TestMisc) [7293570] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:15:59 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293570/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293570/ sentry: https://sentry.ceph.com/organizations/ceph/?query=25fb540c4b094afcb9cf05e8df872f87 Command failed on smithi105 with status 1: 'set -ex\nsudo dd of=/home/ubuntu/cephtest/valgrind.supp' Dead ================================================================= [7293571] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293571/ log: http://qa-proxy.ceph.com/teuthology/neesingh-2023-06-01_06:42:21-fs-wip-neesingh-testing-310523-distro-default-smithi/7293571/ Error reimaging machines: Expected smithi111's OS to be ubuntu 22.04 but found centos 8