Found 222 jobs
Missing teuthology log /home/teuthworker/archive/teuthology-2022-12-23_03:20:03-krbd-quincy-testing-default-smithi/7125948/teuthology.log
Found 10 distinct failure reasons
Failure: {'smithi107.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.581809', 'end': '2022-12-23 04:44:29.005895', 'invocation': {'module_args': {'_raw_params': 'cpan  Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-23 04:44:24.424086', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2506 days old.\n  Please check the host you chose as your CPAN mirror for staleness.\n  I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2506 days old.', '  Please check the host you chose as your CPAN mirror for staleness.', "  I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi110.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.760616', 'end': '2022-12-23 04:44:30.379264', 'invocation': {'module_args': {'_raw_params': 'cpan  Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-23 04:44:25.618648', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2506 days old.\n  Please check the host you chose as your CPAN mirror for staleness.\n  I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2506 days old.', '  Please check the host you chose as your CPAN mirror for staleness.', "  I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}, 'smithi132.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.681659', 'end': '2022-12-23 04:44:30.446023', 'invocation': {'module_args': {'_raw_params': 'cpan  Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-23 04:44:25.764364', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2506 days old.\n  Please check the host you chose as your CPAN mirror for staleness.\n  I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2506 days old.', '  Please check the host you chose as your CPAN mirror for staleness.', "  I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}}
179 jobs
suites intersection: ['conf']
suites union: ['ceph/ceph', 'clusters/3-node', 'clusters/fixed-3', 'clusters/separate-client', 'conf', 'features/no-object-map', 'features/object-map', 'filestore-xfs', 'install/ceph', 'kernels/single-major-off', 'kernels/single-major-on', 'krbd/fsx/{ceph/ceph', 'krbd/ms_modeless/{bluestore-bitmap', 'krbd/rbd-nomount/{bluestore-bitmap', 'krbd/rbd/{bluestore-bitmap', 'krbd/singleton/{bluestore-bitmap', 'krbd/thrash/{bluestore-bitmap', 'krbd/unmap/{ceph/ceph', 'krbd/wac/wac/{bluestore-bitmap', 'ms_mode$/{crc-rxbounce}', 'ms_mode$/{crc}', 'ms_mode$/{legacy-rxbounce}', 'ms_mode$/{legacy}', 'ms_mode$/{secure}', 'ms_mode/crc$/{crc-rxbounce}', 'ms_mode/crc$/{crc}', 'ms_mode/legacy$/{legacy-rxbounce}', 'ms_mode/legacy$/{legacy}', 'ms_mode/secure', 'msgr-failures/few', 'msgr-failures/many', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'randomized-striping-on}', 'striping/default/{msgr-failures/few', 'striping/default/{msgr-failures/many', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-1-client}', 'tasks/fsx-3-client}', 'tasks/krbd_data_pool}', 'tasks/krbd_default_map_options}', 'tasks/krbd_exclusive_option}', 'tasks/krbd_fallocate}', 'tasks/krbd_huge_osdmap}', 'tasks/krbd_latest_osdmap_on_map}', 'tasks/krbd_namespaces}', 'tasks/krbd_rxbounce}', 'tasks/krbd_udev_enumerate}', 'tasks/krbd_udev_netlink_enobufs}', 'tasks/krbd_udev_netns}', 'tasks/krbd_udev_symlinks}', 'tasks/rbd_concurrent}', 'tasks/rbd_fio}', 'tasks/rbd_huge_tickets}', 'tasks/rbd_image_read}', 'tasks/rbd_kernel}', 'tasks/rbd_map_snapshot_io}', 'tasks/rbd_map_unmap}', 'tasks/rbd_simple_big}', 'tasks/rbd_workunit_kernel_untar_build}', 'tasks/rbd_workunit_suites_dbench}', 'tasks/rbd_workunit_suites_ffsb}', 'tasks/rbd_workunit_suites_fsstress_ext4}', 'tasks/rbd_workunit_suites_fsstress}', 'tasks/rbd_workunit_suites_fsx}', 'tasks/rbd_workunit_suites_iozone}', 'tasks/rbd_workunit_trivial_sync}', 'tasks/rbd_xfstests}', 'tasks/unmap}', 'tasks/wac', 'thrashers/backoff', 'thrashers/mon-thrasher', 'thrashers/pggrow', 'thrashers/upmap', 'thrashosds-health', 'verify/many-resets}', 'verify/no-resets}', 'workloads/rbd_fio}', 'workloads/rbd_workunit_suites_ffsb}']

Failure: {'smithi154.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': True, 'cmd': ['cpan', 'Amazon::S3'], 'delta': '0:00:04.714573', 'end': '2022-12-23 04:58:10.701474', 'invocation': {'module_args': {'_raw_params': 'cpan  Amazon::S3', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 25, 'start': '2022-12-23 04:58:05.986901', 'stderr': '', 'stderr_lines': [], 'stdout': "Loading internal logger. Log::Log4perl recommended for better logging\nReading '/home/ubuntu/.cpan/Metadata'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz\nReading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz\nReading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'\n  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT\nWarning: This index file is 2506 days old.\n  Please check the host you chose as your CPAN mirror for staleness.\n  I'll continue but problems seem likely to happen.\x07\n............................................................................DONE\nFetching with LWP:\nhttp://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz\nReading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'\nDONE\nWriting /home/ubuntu/.cpan/Metadata\nRunning install for module 'Amazon::S3'\n\nWarning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.\n\nThe cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.\nProceed nonetheless? [no] no\nAborted.", 'stdout_lines': ['Loading internal logger. Log::Log4perl recommended for better logging', "Reading '/home/ubuntu/.cpan/Metadata'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/authors/01mailrc.txt.gz', "Reading '/home/ubuntu/.cpan/sources/authors/01mailrc.txt.gz'", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/02packages.details.txt.gz', "Reading '/home/ubuntu/.cpan/sources/modules/02packages.details.txt.gz'", '  Database was generated on Fri, 12 Feb 2016 02:17:02 GMT', 'Warning: This index file is 2506 days old.', '  Please check the host you chose as your CPAN mirror for staleness.', "  I'll continue but problems seem likely to happen.\x07", '............................................................................DONE', 'Fetching with LWP:', 'http://apt-mirror.sepia.ceph.com/CPAN/modules/03modlist.data.gz', "Reading '/home/ubuntu/.cpan/sources/modules/03modlist.data.gz'", 'DONE', 'Writing /home/ubuntu/.cpan/Metadata', "Running install for module 'Amazon::S3'", '', "Warning: checksum file '/home/ubuntu/.cpan/sources/authors/id/T/TI/TIMA/CHECKSUMS' not conforming.", '', "The cksum does not contain the key 'cpan_path' for 'Amazon-S3-0.45.tar.gz'.", 'Proceed nonetheless? [no] no', 'Aborted.']}}
24 jobs: ['7125921', '7125938', '7125892', '7125855', '7125774', '7125834', '7125929', '7125884', '7125973', '7125946', '7125909', '7125826', '7125965', '7125871', '7125846', '7125799', '7125809', '7125767', '7125782', '7125817', '7125790', '7125863', '7125759', '7125956']
suites intersection: ['ceph/ceph', 'clusters/fixed-1', 'conf']
suites union: ['ceph/ceph', 'clusters/fixed-1', 'conf', 'krbd/basic/{bluestore-bitmap', 'krbd/wac/sysfs/{bluestore-bitmap', 'ms_mode/crc$/{crc-rxbounce}', 'ms_mode/crc$/{crc}', 'ms_mode/legacy$/{legacy-rxbounce}', 'ms_mode/legacy$/{legacy}', 'ms_mode/secure', 'tasks/krbd_deep_flatten}', 'tasks/krbd_discard}', 'tasks/krbd_huge_image}', 'tasks/krbd_modprobe}', 'tasks/krbd_msgr_segments}', 'tasks/krbd_parent_overlap}', 'tasks/krbd_read_only}', 'tasks/krbd_whole_object_zeroout}', 'tasks/stable_writes}']

Failure: Command failed on smithi160 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main'
4 jobs: ['7125896', '7125813', '7125874', '7125833']
suites intersection: ['clusters/fixed-3', 'conf', 'msgr-failures/many']
suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'krbd/rbd/{bluestore-bitmap', 'ms_mode/crc$/{crc-rxbounce}', 'ms_mode/crc$/{crc}', 'ms_mode/legacy$/{legacy-rxbounce}', 'ms_mode/secure', 'msgr-failures/many', 'tasks/krbd_udev_enumerate}', 'tasks/rbd_concurrent}', 'tasks/rbd_fio}', 'tasks/rbd_simple_big}']

Failure: Cannot connect to remote host smithi160
2 jobs: ['7125810', '7125827']
suites intersection: ['clusters/3-node', 'conf', 'features/object-map', 'krbd/fsx/{ceph/ceph', 'tasks/fsx-3-client}']
suites union: ['clusters/3-node', 'conf', 'features/object-map', 'krbd/fsx/{ceph/ceph', 'ms_mode$/{legacy}', 'ms_mode$/{secure}', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'randomized-striping-on}', 'striping/default/{msgr-failures/many', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-3-client}']

Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=nightly_pre-single-major
1 jobs: ['7125766']
suites: ['clusters/separate-client', 'conf', 'filestore-xfs', 'kernels/pre-single-major', 'krbd/unmap/{ceph/ceph', 'tasks/unmap}']

Dead: None
7 jobs: ['7125948', '7125859', '7125796', '7125792', '7125779', '7125778', '7125793']

Failure: Stale jobs detected, aborting.
1 jobs: ['7125795']
suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/secure', 'msgr-failures/few', 'tasks/rbd_map_snapshot_io}']

Failure: SSH connection to smithi160 was lost: 'rm -f /tmp/linux-image.deb && echo linux-image-6.1.0-ceph-g937d5e0c86d7_6.1.0-ceph-g937d5e0c86d7-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/937d5e0c86d70ca5fb80d65cdf2379d67b3bb37b/ubuntu/jammy/flavors/default/pool/main/l/linux-6.1.0-ceph-g937d5e0c86d7/ --input-file=-'
2 jobs: ['7125900', '7125805']
suites intersection: ['conf']
suites union: ['ceph/ceph', 'clusters/fixed-1', 'clusters/fixed-3', 'conf', 'install/ceph', 'krbd/basic/{bluestore-bitmap', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/crc$/{crc}', 'ms_mode/legacy$/{legacy}', 'msgr-failures/many', 'tasks/krbd_fallocate}', 'tasks/krbd_whole_object_zeroout}']

Failure: SSH connection to smithi160 was lost: 'sudo dpkg -i /tmp/linux-image.deb'
1 jobs: ['7125869']
suites: ['clusters/fixed-3', 'conf', 'krbd/rbd/{bluestore-bitmap', 'ms_mode/crc$/{crc-rxbounce}', 'msgr-failures/few', 'tasks/rbd_workunit_trivial_sync}']

Failure: 
1 jobs: ['7125816']
suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/legacy$/{legacy}', 'msgr-failures/many', 'tasks/krbd_udev_netns}']