2022-07-26T12:32:29.200 INFO:root:teuthology version: 1.1.1.dev392+gefc3ff73
2022-07-26T12:32:29.201 INFO:teuthology.results:subset: '21/32'
2022-07-26T12:32:29.201 INFO:teuthology.results:seed: '5756'
2022-07-26T12:33:28.257 INFO:root:teuthology version: 1.1.1.dev392+gefc3ff73
2022-07-26T12:33:28.261 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-07-26T19:41:07.622 INFO:teuthology.results:Tests finished! gathering results...
2022-07-26T19:41:11.667 INFO:scrape:Found 40 jobs
2022-07-26T19:41:18.405 INFO:scrape:Found 6 distinct failure reasons
2022-07-26T19:41:18.406 INFO:scrape:Failure: "2022-07-26T13:14:30.741951+0000 mds.a (mds.0) 25 : cluster [WRN] client.4756 isn't responding to mclientcaps(revoke), ino 0x100000051c8 pending pFc issued pFcb, sent 300.021574 seconds ago" in cluster log
2022-07-26T19:41:18.407 INFO:scrape:2 jobs: ['6947475', '6947450']
2022-07-26T19:41:18.408 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}']
2022-07-26T19:41:18.408 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}']
2022-07-26T19:41:18.409 INFO:scrape:
2022-07-26T19:41:18.410 INFO:scrape:Failure: {'smithi156.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection refused>'}, 'smithi064.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection refused>'}, 'smithi038.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection refused>'}}
2022-07-26T19:41:18.410 INFO:scrape:29 jobs: ['6947449', '6947467', '6947454', '6947436', '6947466', '6947459', '6947444', '6947474', '6947470', '6947462', '6947464', '6947439', '6947442', '6947455', '6947447', '6947469', '6947472', '6947460', '6947473', '6947453', '6947451', '6947461', '6947468', '6947463', '6947441', '6947456', '6947457', '6947440', '6947458']
2022-07-26T19:41:18.411 INFO:scrape:suites intersection: ['2-logrotate}', 'conf/{client', 'mds', 'mon', 'osd}']
2022-07-26T19:41:18.412 INFO:scrape:suites union: ['1-ceph', '1-cephadm', '2-logrotate}', '2-workunit/fs/snaps}}', '2-workunit/suites/fsstress}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'fs/multifs/{begin/{0-install', 'fs/snaps/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'fs/verify/{begin/{0-install', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mon-debug', 'mon-debug}', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/legacy', 'ms_mode/secure', 'msgr-failures/osd-mds-delay', 'n/3', 'n/5', 'no_client_pidfile}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'overrides/{ignorelist_health', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'ranks/multi/{export-check', 'replication/always}', 'replication/default}', 'rhel_8}', 'scrub/no', 'scrub/yes', 'session_timeout', 'session_timeout}', 'standby-replay', 'subvolume/{no-subvolume}', 'subvolume/{with-namespace-isolated-and-quota}', 'subvolume/{with-namespace-isolated}', 'subvolume/{with-no-extra-options}', 'subvolume/{with-quota}', 'tasks/client-recovery}', 'tasks/dbench', 'tasks/failover}', 'tasks/workunit/snaps}', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/osd', 'thrashosds-health}', 'validater/valgrind}', 'workunit/fs/misc}}', 'workunit/suites/blogbench}}', 'workunit/suites/dbench}}', 'workunit/suites/fsync-tester}}', 'workunit/suites/pjd}}', 'wsync/no}', 'wsync/yes}']
2022-07-26T19:41:18.412 INFO:scrape:
2022-07-26T19:41:18.413 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2022-07-26T19:41:18.413 INFO:scrape:1 jobs: ['6947471']
2022-07-26T19:41:18.414 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/1', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}']
2022-07-26T19:41:18.414 INFO:scrape:
2022-07-26T19:41:18.415 INFO:scrape:Failure: "2022-07-26T13:09:16.120035+0000 mon.a (mon.0) 938 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
2022-07-26T19:41:18.416 INFO:scrape:1 jobs: ['6947446']
2022-07-26T19:41:18.417 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2022-07-26T19:41:18.417 INFO:scrape:
2022-07-26T19:41:18.418 INFO:scrape:Failure: "2022-07-26T18:52:46.619441+0000 osd.4 (osd.4) 96 : cluster [ERR] 5.1cs0 deep-scrub : stat mismatch, got 0/1 objects, 0/0 clones, 0/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 0/2000 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log
2022-07-26T19:41:18.419 INFO:scrape:1 jobs: ['6947445']
2022-07-26T19:41:18.420 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}']
2022-07-26T19:41:18.421 INFO:scrape:
2022-07-26T19:41:18.422 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json
2022-07-26T19:41:18.423 INFO:scrape:1 jobs: ['6947438']
2022-07-26T19:41:18.424 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'races', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}']
2022-07-26T19:41:18.424 INFO:scrape:
2022-07-26T19:41:18.428 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/
failed:  6
dead:    29
running: 0
waiting: 0
queued:  0
passed:  5


Fail
=================================================================
[6947438]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   03:38:24
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947438/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947438/

    Command failed on smithi019 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph pg dump --format=json'

[6947445]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   06:59:27
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947445/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947445/

    "2022-07-26T18:52:46.619441+0000 osd.4 (osd.4) 96 : cluster [ERR] 5.1cs0
    deep-scrub : stat mismatch, got 0/1 objects, 0/0 clones, 0/1 dirty, 0/0
    omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 0/2000 bytes, 0/0
    manifest objects, 0/0 hit_set_archive bytes." in cluster log

[6947446]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:50:19
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947446/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947446/

    "2022-07-26T13:09:16.120035+0000 mon.a (mon.0) 938 : cluster [WRN] Health
    check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)"
    in cluster log

[6947450]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   06:13:48
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947450/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947450/

    "2022-07-26T13:28:00.420729+0000 mds.c (mds.0) 1 : cluster [WRN]
    client.4887 isn't responding to mclientcaps(revoke), ino 0x1000000313a
    pending pAsLsXsFscr issued pAsLsXsFsxcrwb, sent 300.008434 seconds ago" in
    cluster log

[6947471]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   00:26:26
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947471/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947471/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bd4fbdfbb2b047a6af2dac927a94d89a

    reached maximum tries (90) after waiting for 540 seconds

[6947475]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   04:56:22
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947475/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947475/

    "2022-07-26T13:14:30.741951+0000 mds.a (mds.0) 25 : cluster [WRN]
    client.4756 isn't responding to mclientcaps(revoke), ino 0x100000051c8
    pending pFc issued pFcb, sent 300.021574 seconds ago" in cluster log



Dead
=================================================================
[6947436]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:08:07
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947436/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947436/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3afc7d60804947a3a0c7bac6db29de91

    {'smithi003.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi049.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi066.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947439]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:59
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947439/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947439/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f66853b074ea4a9f9ed561f10c8a2819

    {'smithi114.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi183.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi152.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947440]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/dbench}}
-----------------------------------------------------------------
time:   00:09:43
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947440/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947440/
sentry: https://sentry.ceph.com/organizations/ceph/?query=94e73811ae614d4f89936f9b623133e9

    {'smithi111.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi138.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi050.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947441]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   00:07:25
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947441/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947441/
sentry: https://sentry.ceph.com/organizations/ceph/?query=229c2631bb5b4f5f944aa1e46c70830a

    {'smithi159.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi146.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947442]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:04:39
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947442/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947442/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7821795836c345c8a25cd93d8caa64e0

    {'smithi052.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi105.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi187.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947444]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:07:58
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947444/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947444/
sentry: https://sentry.ceph.com/organizations/ceph/?query=142fad815b0a4602a4d383d5edfad4d4

    {'smithi080.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi022.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi125.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947447]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:27
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947447/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947447/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a7e221a941a246dc819c2207c3294011

    {'smithi158.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi192.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi188.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947449]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/dbench}}
-----------------------------------------------------------------
time:   00:09:12
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947449/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947449/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4297fc6f82d244dc9327dda52d3002ef

    {'smithi156.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi064.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi038.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947451]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:09:11
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947451/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947451/
sentry: https://sentry.ceph.com/organizations/ceph/?query=66dc782c2fb04a839aaaab0b8e2d3d0b

    {'smithi033.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi202.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi043.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947453]  fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
-----------------------------------------------------------------
time:   00:08:30
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947453/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947453/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b53b41b1d1e34c38a839e5b1a4735eed

    {'smithi145.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi073.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947454]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:09:11
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947454/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947454/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f09d720cbf354f17b494853c10f95e7f

    {'smithi085.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi017.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi057.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947455]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} scrub/yes standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/fs/misc}}
-----------------------------------------------------------------
time:   00:08:51
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947455/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947455/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ceaba4a851ec457880c677de41e0faec

    {'smithi055.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi197.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi112.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947456]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
-----------------------------------------------------------------
time:   00:06:50
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947456/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947456/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3f2d6dda600f423e803cfd6aee0403eb

    {'smithi083.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi120.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947457]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsync-tester}}
-----------------------------------------------------------------
time:   00:08:41
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947457/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947457/
sentry: https://sentry.ceph.com/organizations/ceph/?query=06e0d6b31aa24c77a4cfb93b4bbdfc79

    {'smithi029.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi119.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi136.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947458]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:09:04
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947458/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947458/
sentry: https://sentry.ceph.com/organizations/ceph/?query=34b4916535144fcebca686ee737cb637

    {'smithi031.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi074.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi115.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947459]  fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
-----------------------------------------------------------------
time:   00:07:30
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947459/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947459/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7fad335d7b864b8c8e77d91937553632

    {'smithi181.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi098.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947460]  fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:08:00
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947460/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947460/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a61376548a964c399e8818ba44afb9d2

    {'smithi139.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi032.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947461]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} scrub/yes standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:14
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947461/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947461/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3c11c38262824b5580bfe326d0567e66

    {'smithi045.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi176.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi117.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947462]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
-----------------------------------------------------------------
time:   00:05:01
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947462/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947462/
sentry: https://sentry.ceph.com/organizations/ceph/?query=186db992a93e4abcb4cf2a893050dba2

    {'smithi172.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi035.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947463]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} scrub/yes standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:08:44
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947463/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947463/
sentry: https://sentry.ceph.com/organizations/ceph/?query=83c35595e443419cb75417d869963c50

    {'smithi101.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi135.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi142.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947464]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:12
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947464/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947464/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e09d2a07bc514e5dac450f327d891d95

    {'smithi002.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi171.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi007.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947466]  fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
-----------------------------------------------------------------
time:   00:07:28
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947466/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947466/
sentry: https://sentry.ceph.com/organizations/ceph/?query=65663216fa36448aa564caef783b03e4

    {'smithi070.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi079.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947467]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:09:06
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947467/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947467/
sentry: https://sentry.ceph.com/organizations/ceph/?query=99ad9ba8dac54f9abd5ced61885c8ee4

    {'smithi037.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi081.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi062.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947468]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:48
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947468/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947468/
sentry: https://sentry.ceph.com/organizations/ceph/?query=415b9b5972f14149bcbe9fca9228d47d

    {'smithi122.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi190.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi170.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947469]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
-----------------------------------------------------------------
time:   00:07:30
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947469/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947469/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0ed7f7501e644dfe83c651e9698c8503

    {'smithi040.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi097.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947470]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} scrub/yes standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:07:47
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947470/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947470/
sentry: https://sentry.ceph.com/organizations/ceph/?query=53d84cf76f8c4b8bacafd5b84a433f4f

    {'smithi006.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi153.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi204.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947472]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/dbench}}
-----------------------------------------------------------------
time:   00:08:21
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947472/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947472/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7754f8e31e9e4a6594a1a11053cd0be4

    {'smithi157.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi150.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi036.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947473]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/pjd}}
-----------------------------------------------------------------
time:   00:09:04
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947473/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947473/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dc9bd3c2ba3f4d519b8acaadbeb461d6

    {'smithi086.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi087.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi174.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}

[6947474]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} scrub/yes standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/suites/blogbench}}
-----------------------------------------------------------------
time:   00:08:28
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947474/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947474/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dee6eb3d7b1645daafa8c90933889987

    {'smithi044.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi141.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}, 'smithi163.front.sepia.ceph.com': {'_ansible_no_log': False,
    'changed': False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': True,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name':
    ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken':
    False, 'state': 'present', 'update_cache': False, 'update_only': False,
    'validate_certs': False}}, 'msg': 'Failure downloading
    http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-
    latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection
    refused>'}}



Pass
=================================================================
[6947437] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:54:40
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947437/

[6947443] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   03:34:55
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947443/

[6947448] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   03:18:43
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947448/

[6947452] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}}
time:   02:31:42
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947452/

[6947465] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment}
time:   00:21:20
info:   http://pulpito.ceph.com/pdonnell-2022-07-26_12:32:13-fs-wip-pdonnell-testing-20220725.202529-distro-default-smithi/6947465/