2026-02-12T16:40:48.910 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T16:40:48.910 INFO:teuthology.results:subset: '562/1024' 2026-02-12T16:40:48.910 INFO:teuthology.results:seed: '4311' 2026-02-12T16:45:46.210 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T16:45:46.217 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2026-02-12T17:46:50.173 INFO:teuthology.results:Tests finished! gathering results... 2026-02-12T17:46:50.454 INFO:scrape:Found 10 jobs 2026-02-12T17:46:51.487 INFO:scrape:Found 3 distinct failure reasons 2026-02-12T17:46:51.487 INFO:scrape:Failure: Fuse mount failed to populate/sys/ after 31 seconds 2026-02-12T17:46:51.487 INFO:scrape:1 jobs: ['46847'] 2026-02-12T17:46:51.487 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/referent}'] 2026-02-12T17:46:51.488 INFO:scrape: 2026-02-12T17:46:51.488 INFO:scrape:Failure: Command failed on trial048 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback' 2026-02-12T17:46:51.488 INFO:scrape:3 jobs: ['46848', '46853', '46846'] 2026-02-12T17:46:51.488 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}'] 2026-02-12T17:46:51.488 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:functional/{0-use-snaprealm/{global}', 'fs:functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/backtrace}', 'tasks/quiesce}', 'tasks/snap_schedule_snapdir}'] 2026-02-12T17:46:51.488 INFO:scrape: 2026-02-12T17:46:51.488 INFO:scrape:Failure: Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration.test_journal_migration) 2026-02-12T17:46:51.488 INFO:scrape:1 jobs: ['46849'] 2026-02-12T17:46:51.488 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs:functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/test_journal_migration}'] 2026-02-12T17:46:51.488 INFO:scrape: 2026-02-12T17:46:51.488 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial ================================================================= info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/ logs: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 5 Fail ================================================================= [46846] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quiesce} ----------------------------------------------------------------- time: 00:08:45 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46846/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46846/ Command failed on trial031 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [46847] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/referent} ----------------------------------------------------------------- time: 00:05:31 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46847/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46847/ Fuse mount failed to populate/sys/ after 31 seconds [46848] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} ----------------------------------------------------------------- time: 00:08:17 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46848/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46848/ Command failed on trial048 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/et c/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback' [46849] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:05:46 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46849/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46849/ Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.T estJournalMigration.test_journal_migration) [46853] fs:functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/backtrace} ----------------------------------------------------------------- time: 00:07:53 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46853/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46853/ Command failed on trial034 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' Pass ================================================================= [46844] fs:functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-full} time: 00:28:16 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46844/ [46845] fs:functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} time: 00:19:04 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46845/ [46850] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/uninlining} time: 00:55:39 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46850/ [46851] fs:functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/quota} time: 00:04:16 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46851/ [46852] fs:functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} time: 00:04:54 info: https://pulpito.ceph.com/pdonnell-2026-02-12_16:38:03-fs:functional-wip-pdonnell-testing-20260210.171210-distro-default-trial/46852/ 2026-02-12T17:46:51.514 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 33, in main results(args['--archive-dir'], args['--name'], args['--email'], File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 78, in results email_results( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 95, in email_results smtp = smtplib.SMTP('localhost') ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 255, in __init__ (code, msg) = self.connect(host, port) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 341, in connect self.sock = self._get_socket(host, port, self.timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket return socket.create_connection((host, port), timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection sock.connect(sa) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect self._internal_connect(address) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect raise _SocketError(err, strerror(err)) ConnectionRefusedError: [Errno 111] Connection refused