2026-02-25T13:25:20.742 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-25T13:25:20.743 INFO:teuthology.results:subset: '883/1024'
2026-02-25T13:25:20.743 INFO:teuthology.results:seed: '56'
2026-02-25T13:29:26.527 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-25T13:29:26.533 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2026-02-25T14:09:34.216 INFO:teuthology.results:Tests finished! gathering results...
2026-02-25T14:09:34.455 INFO:scrape:Found 6 jobs
2026-02-25T14:09:34.774 INFO:scrape:Found 1 distinct failure reasons
2026-02-25T14:09:34.774 INFO:scrape:Failure: Command failed on trial034 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0'
2026-02-25T14:09:34.774 INFO:scrape:6 jobs: ['70202', '70198', '70197', '70201', '70200', '70199']
2026-02-25T14:09:34.775 INFO:scrape:suites intersection: ['2-logrotate}', '2-mount/{ganesha-client', '3-workload/iogen}}', 'centos_9.stream', 'clusters/1a3s-mds-1c', 'conf/{client', 'fs:nfs-ganesha/{begin/{0-install', 'ganesha', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/{0-create-export']
2026-02-25T14:09:34.775 INFO:scrape:suites union: ['1-apply-config/{async/no', '1-apply-config/{async/yes', '2-logrotate}', '2-mount/{ganesha-client', '3-workload/iogen}}', 'centos_9.stream', 'clusters/1a3s-mds-1c', 'conf/{client', 'delegations/no', 'delegations/rw', 'fs:nfs-ganesha/{begin/{0-install', 'ganesha', 'mds', 'mgr', 'mon', 'nfs-version/4.1}', 'nfs-version/4.2}', 'nfs-version/latest}', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/{0-create-export', 'zerocopy/no}', 'zerocopy/yes}']
2026-02-25T14:09:34.775 INFO:scrape:
2026-02-25T14:09:34.775 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial
=================================================================
info:    https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/
logs:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[70197]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/no delegations/rw ganesha zerocopy/no} 2-mount/{ganesha-client nfs-version/4.1} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:31
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70197/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70197/

    Command failed on trial020 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'

[70198]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/yes delegations/no ganesha zerocopy/yes} 2-mount/{ganesha-client nfs-version/4.2} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:21
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70198/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70198/

    Command failed on trial046 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'

[70199]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/yes delegations/no ganesha zerocopy/no} 2-mount/{ganesha-client nfs-version/latest} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:35
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70199/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70199/

    Command failed on trial049 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'

[70200]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/no delegations/rw ganesha zerocopy/yes} 2-mount/{ganesha-client nfs-version/4.1} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:10
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70200/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70200/

    Command failed on trial038 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'

[70201]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/yes delegations/rw ganesha zerocopy/no} 2-mount/{ganesha-client nfs-version/4.2} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:55
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70201/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70201/

    Command failed on trial102 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'

[70202]  fs:nfs-ganesha/{begin/{0-install 2-logrotate} centos_9.stream clusters/1a3s-mds-1c conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} tasks/{0-create-export 1-apply-config/{async/no delegations/no ganesha zerocopy/yes} 2-mount/{ganesha-client nfs-version/latest} 3-workload/iogen}}
-----------------------------------------------------------------
time:   00:24:52
info:   https://pulpito.ceph.com/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70202/
log:    https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-25_12:55:41-fs:nfs-ganesha-wip-vshankar-testing-20260225.064457-testing-default-trial/70202/

    Command failed on trial034 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0'
2026-02-25T14:09:34.807 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 33, in main
    results(args['--archive-dir'], args['--name'], args['--email'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 78, in results
    email_results(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 95, in email_results
    smtp = smtplib.SMTP('localhost')
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection
    sock.connect(sa)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect
    self._internal_connect(address)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect
    raise _SocketError(err, strerror(err))
ConnectionRefusedError: [Errno 111] Connection refused