2022-08-09T17:43:28.311 INFO:root:teuthology version: 1.1.1.dev407+g9e4bf63f 2022-08-09T17:43:28.313 INFO:teuthology.results:subset: '17/32' 2022-08-09T17:43:28.313 INFO:teuthology.results:seed: '3993' 2022-08-21T09:46:54.982 INFO:root:teuthology version: 1.1.1.dev412+gb1d387f1 2022-08-21T09:46:55.013 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-08-21T21:40:56.691 INFO:teuthology.results:Tests finished! gathering results... 2022-08-21T21:41:24.639 INFO:scrape:Found 292 jobs 2022-08-21T21:41:38.041 INFO:scrape:Found 14 distinct failure reasons 2022-08-21T21:41:38.043 INFO:scrape:Failure: unexpected end of data 2022-08-21T21:41:38.043 INFO:scrape:1 jobs: ['6962951'] 2022-08-21T21:41:38.044 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'fs/traceless/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{frag', 'tasks/cfuse_workunit_suites_blogbench', 'traceless/50pc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.044 INFO:scrape: 2022-08-21T21:41:38.045 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2022-08-21T21:41:38.045 INFO:scrape:2 jobs: ['6963063', '6962940'] 2022-08-21T21:41:38.045 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'mds', 'mon', 'osd}', 'ranks/5', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.046 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu/{latest', 'fs/thrash/workloads/{begin/{0-install', 'fs/verify/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag', 'overrides/{mon-debug', 'overrides}}', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/fsstress', 'tasks/{1-thrash/osd', 'thrashosds-health', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.047 INFO:scrape: 2022-08-21T21:41:38.047 INFO:scrape:Failure: {'smithi103.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: '}, 'smithi141.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: '}} 2022-08-21T21:41:38.047 INFO:scrape:2 jobs: ['6962848', '6962869'] 2022-08-21T21:41:38.048 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/stock/{k-stock', 'rhel_8}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.048 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/suites/iozone}}', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'overrides/{no_client_pidfile', 'prefetch_dirfrags/no', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/mds_creation_retry}', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.049 INFO:scrape: 2022-08-21T21:41:38.049 INFO:scrape:Failure: "2022-08-09T19:09:32.032045+0000 mon.a (mon.0) 2235 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log 2022-08-21T21:41:38.050 INFO:scrape:3 jobs: ['6962834', '6962930', '6962804'] 2022-08-21T21:41:38.050 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{no_client_pidfile', 'rhel_8}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.051 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/client-recovery}', 'tasks/exports}', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.051 INFO:scrape: 2022-08-21T21:41:38.051 INFO:scrape:Failure: {'smithi080.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['gdisk', 'device-mapper'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi183.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} 2022-08-21T21:41:38.052 INFO:scrape:7 jobs: ['6962841', '6962839', '6962867', '6962842', '6962835', '6962861', '6962857'] 2022-08-21T21:41:38.053 INFO:scrape:suites intersection: ['2-logrotate}', 'conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.053 INFO:scrape:suites union: ['1-ceph', '1-cephadm', '2-logrotate}', '2-workunit/suites/ffsb}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{secure}', 'msgr-failures/none', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'overrides/{no_client_pidfile', 'prefetch_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/mds-flush}', 'tasks/{0-check-counter', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'workunit/fs/test_o_trunc}', 'workunit/suites/blogbench}', 'workunit/suites/fsync-tester}', 'workunit/suites/iogen}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-08-21T21:41:38.053 INFO:scrape: 2022-08-21T21:41:38.054 INFO:scrape:Dead: b'2022-08-21T21:23:02.329 DEBUG:teuthology.exit:Finished running handlers' 2022-08-21T21:41:38.054 INFO:scrape:2 jobs: ['6963058', '6962990'] 2022-08-21T21:41:38.055 INFO:scrape:suites intersection: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-ec-root', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'ranks/3', 'rhel_8}', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'wsync/{no}}'] 2022-08-21T21:41:38.055 INFO:scrape:suites union: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'ranks/3', 'rhel_8}', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}', 'workunit/suites/blogbench}', 'wsync/{no}}'] 2022-08-21T21:41:38.056 INFO:scrape: 2022-08-21T21:41:38.056 INFO:scrape:Crash: "2022-08-21T10:18:52.135639+0000 mon.a (mon.0) 1690 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log 2022-08-21T21:41:38.056 INFO:scrape:ceph version 17.2.3-383-ge4a15b78 (e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc) quincy (stable) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f0d13f30b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7f0d14f4cd72] 5: /usr/lib64/ceph/libceph-common.so.2(+0x283f35) [0x7f0d14f4cf35] 6: ceph-mds(+0x28b6a3) [0x56470519a6a3] 7: (MDCache::dispatch_fragment_dir(boost::intrusive_ptr&)+0x4f9) [0x56470522c389] 8: (MDCache::fragment_frozen(boost::intrusive_ptr&, int)+0x2d8) [0x56470522d1c8] 9: (MDCache::fragment_mark_and_complete(boost::intrusive_ptr&)+0xa3c) [0x56470523df2c] 10: (MDCache::merge_dir(CInode*, frag_t)+0x5ab) [0x56470523ed6b] 11: ceph-mds(+0x411d02) [0x564705320d02] 12: (Context::complete(int)+0xd) [0x5647050da2bd] 13: (CommonSafeTimer::timer_thread()+0x18b) [0x7f0d1506080b] 14: (CommonSafeTimerThread::entry()+0x11) [0x7f0d15062811] 15: /lib64/libpthread.so.0(+0x814a) [0x7f0d13f2614a] 16: clone() 2022-08-21T21:41:38.057 INFO:scrape:1 jobs: ['6963047'] 2022-08-21T21:41:38.057 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.058 INFO:scrape: 2022-08-21T21:41:38.058 INFO:scrape:Failure: {'smithi195.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'cmd': "subscription-manager release --list | grep -E '[0-9]'", 'delta': '0:00:02.655349', 'end': '2022-08-09 18:50:52.445655', 'failed_when_result': True, 'invocation': {'module_args': {'_raw_params': "subscription-manager release --list | grep -E '[0-9]'", '_uses_shell': True, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 1, 'start': '2022-08-09 18:50:49.790306', 'stderr': 'Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.', 'stderr_lines': ['Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.'], 'stdout': '', 'stdout_lines': []}, 'smithi105.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-08-21T21:41:38.059 INFO:scrape:1 jobs: ['6962845'] 2022-08-21T21:41:38.059 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/trivial_sync}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/none', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.060 INFO:scrape: 2022-08-21T21:41:38.060 INFO:scrape:Failure: {'smithi185.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi160.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-08-21T21:41:38.060 INFO:scrape:1 jobs: ['6962844'] 2022-08-21T21:41:38.061 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{no_client_pidfile', 'tasks/fragment}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.061 INFO:scrape: 2022-08-21T21:41:38.062 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh' 2022-08-21T21:41:38.062 INFO:scrape:1 jobs: ['6963027'] 2022-08-21T21:41:38.062 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.063 INFO:scrape: 2022-08-21T21:41:38.063 INFO:scrape:Failure: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2022-08-21T21:41:38.064 INFO:scrape:1 jobs: ['6963002'] 2022-08-21T21:41:38.064 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.065 INFO:scrape: 2022-08-21T21:41:38.065 INFO:scrape:Failure: Command failed on smithi086 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' 2022-08-21T21:41:38.065 INFO:scrape:1 jobs: ['6962864'] 2022-08-21T21:41:38.066 INFO:scrape:suites: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount/fuse', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag', 'ranks/1', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}', 'wsync/{no}}'] 2022-08-21T21:41:38.066 INFO:scrape: 2022-08-21T21:41:38.067 INFO:scrape:Crash: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi130 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2022-08-21T21:41:38.067 INFO:scrape:ceph version 17.2.3-383-ge4a15b78 (e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc) quincy (stable) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f4b8bcee420] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f4b8bf79834] 5: /usr/lib/ceph/libceph-common.so.2(+0x27c9de) [0x7f4b8bf799de] 6: (std::_Sp_counted_ptr::_M_dispose()+0x1cd) [0x563c885e84dd] 7: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x48) [0x563c885e16c8] 8: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x563c885c9898] 9: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x513) [0x563c885d0473] 10: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f4b8c213020] 11: (DispatchQueue::entry()+0x58f) [0x7f4b8c2108bf] 12: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f4b8c2df701] 13: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f4b8bce2609] 14: clone() 2022-08-21T21:41:38.068 INFO:scrape:1 jobs: ['6962858'] 2022-08-21T21:41:38.068 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-08-21T21:41:38.068 INFO:scrape: 2022-08-21T21:41:38.069 INFO:scrape:Failure: Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) 2022-08-21T21:41:38.069 INFO:scrape:1 jobs: ['6962826'] 2022-08-21T21:41:38.070 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}'] 2022-08-21T21:41:38.070 INFO:scrape: 2022-08-21T21:41:38.084 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/ failed: 12 dead: 13 running: 0 waiting: 0 queued: 0 passed: 267 Fail ================================================================= [6962804] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:30:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962804/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962804/ "2022-08-09T18:06:58.878075+0000 mon.a (mon.0) 605 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [6962826] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} ----------------------------------------------------------------- time: 02:08:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962826/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962826/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3fb3cbe0f7494236809929fb00e2cc33 Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) [6962834] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} ----------------------------------------------------------------- time: 00:45:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962834/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962834/ "2022-08-09T19:09:32.032045+0000 mon.a (mon.0) 2235 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [6962858] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:02:11 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962858/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962858/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b6123c140d5d4875a888d9db3d1b0f86 Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi130 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6962864] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} ----------------------------------------------------------------- time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962864/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962864/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9af3ce7a8ba04238aafc52ddcc768b6c Command failed on smithi086 with status 1: 'TESTDIR=/home/ubuntu/cephtest bash -s' [6962930] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:46:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962930/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962930/ "2022-08-10T03:18:32.477790+0000 mon.a (mon.0) 420 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6962940] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:20:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962940/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962940/ sentry: https://sentry.ceph.com/organizations/ceph/?query=976c3c2c43bb40899929ff98b3fc2cbd reached maximum tries (90) after waiting for 540 seconds [6962951] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962951/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962951/ unexpected end of data [6963002] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 04:18:53 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963002/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963002/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0420a9ce87d64551827df577cf8d7110 Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6963027] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:42:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963027/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963027/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b9aafa8934be4f9a8859f24dcb1a1b43 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4a15b78f09bd71cdbe245e9a0ac083d2efb82dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6963047] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:45:42 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963047/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963047/ "2022-08-21T10:18:52.135639+0000 mon.a (mon.0) 1690 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log [6963063] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 01:44:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963063/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963063/ reached maximum tries (50) after waiting for 300 seconds Dead ================================================================= [6962835] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962835/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962835/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ffbb869201cd4f2196f4863267b10ab6 {'smithi202.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph- selinux', 'ceph-common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python- cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi143.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph-selinux', 'ceph- common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python-cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962839] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} ----------------------------------------------------------------- time: 00:08:17 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962839/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962839/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bc47d69f1454406929242518cf3f99b {'smithi138.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi187.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi130.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962841] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962841/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962841/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c56a307925ec4312bda9e7bffe44d009 {'smithi080.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['gdisk', 'device-mapper'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi183.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962842] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} ----------------------------------------------------------------- time: 00:06:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962842/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962842/ sentry: https://sentry.ceph.com/organizations/ceph/?query=95d1ca40516a4960b5e3dcd59cee3e98 {'smithi142.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['python3-libsemanage', 'python3-policycoreutils'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'installed', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi116.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph-selinux', 'ceph- common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python-cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi085.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph-selinux', 'ceph- common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python-cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962844] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} ----------------------------------------------------------------- time: 00:04:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962844/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962844/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fc77b19de9354939ace9017bd1a78e66 {'smithi185.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi160.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6962845] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:04:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962845/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962845/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a103e35677e84f35bc00a85a0c32866b {'smithi195.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'cmd': "subscription-manager release --list | grep -E '[0-9]'", 'delta': '0:00:02.655349', 'end': '2022-08-09 18:50:52.445655', 'failed_when_result': True, 'invocation': {'module_args': {'_raw_params': "subscription-manager release --list | grep -E '[0-9]'", '_uses_shell': True, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'msg': 'non-zero return code', 'rc': 1, 'start': '2022-08-09 18:50:49.790306', 'stderr': 'Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.', 'stderr_lines': ['Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.'], 'stdout': '', 'stdout_lines': []}, 'smithi105.front.sepia.ceph.com': {'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6962848] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:03:26 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962848/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962848/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cb077ba92c9a4c2fa584c7c9843470ed {'smithi103.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm, Request failed: '}, 'smithi141.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm, Request failed: '}} [6962857] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} ----------------------------------------------------------------- time: 00:12:11 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962857/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962857/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7d3abe26774a4a07954bd6dfede0aa2d {'smithi105.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['ceph', 'ceph-base', 'ceph- selinux', 'ceph-common', 'ceph-debuginfo', 'ceph-release', 'libcephfs1', 'ceph-radosgw', 'python-ceph', 'python-rados', 'python-rbd', 'python- cephfs', 'librbd1', 'librados2', 'mod_fastcgi'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962861] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962861/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962861/ sentry: https://sentry.ceph.com/organizations/ceph/?query=97a026031f2f45e984f7e5c3cb8beeb3 {'smithi072.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi183.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['gdisk', 'device-mapper'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi099.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['gdisk', 'device-mapper'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962867] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962867/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962867/ sentry: https://sentry.ceph.com/organizations/ceph/?query=acffaf2ca3aa4157b493b21a2bcfaefd {'smithi079.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi135.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi179.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['podman', 'podman-docker'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'latest', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}} [6962869] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} ----------------------------------------------------------------- time: 00:03:12 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962869/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962869/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b11ef491d984a5c993beccdb188f022 {'smithi136.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm, Request failed: '}, 'smithi201.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': True, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': False}}, 'msg': 'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm, Request failed: '}} [6962990] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962990/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962990/ hit max job timeout [6963058] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963058/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963058/ hit max job timeout Pass ================================================================= [6962799] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962799/ [6962800] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962800/ [6962801] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:01:07 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962801/ [6962802] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:22:09 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962802/ [6962803] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962803/ [6962805] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:48:32 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962805/ [6962806] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 00:59:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962806/ [6962807] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:41:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962807/ [6962808] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:31:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962808/ [6962809] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:28:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962809/ [6962810] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:24:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962810/ [6962811] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:35 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962811/ [6962812] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962812/ [6962813] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} time: 00:48:02 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962813/ [6962814] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full} time: 00:14:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962814/ [6962815] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962815/ [6962816] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} time: 01:57:30 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962816/ [6962817] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} time: 00:25:22 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962817/ [6962818] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:49:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962818/ [6962819] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:52:35 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962819/ [6962820] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:13:21 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962820/ [6962821] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:36:01 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962821/ [6962822] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:47:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962822/ [6962823] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:14:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962823/ [6962824] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{rhel_8} tasks/fstop} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962824/ [6962825] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:44 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962825/ [6962827] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/lockdep} time: 00:53:55 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962827/ [6962828] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:20:10 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962828/ [6962829] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:28:14 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962829/ [6962830] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:29:39 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962830/ [6962831] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962831/ [6962832] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:28:53 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962832/ [6962833] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962833/ [6962836] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:18:36 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962836/ [6962837] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962837/ [6962838] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:29:22 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962838/ [6962840] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:21:58 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962840/ [6962843] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:30:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962843/ [6962846] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:35:42 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962846/ [6962847] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:12:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962847/ [6962849] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:37:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962849/ [6962850] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:55:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962850/ [6962851] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:27:43 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962851/ [6962852] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:21:14 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962852/ [6962853] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:21:39 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962853/ [6962854] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:26:34 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962854/ [6962855] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:26:07 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962855/ [6962856] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:21:24 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962856/ [6962859] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962859/ [6962860] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:24 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962860/ [6962862] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:41:14 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962862/ [6962863] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:23:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962863/ [6962865] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:28:08 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962865/ [6962866] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:16:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962866/ [6962868] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:05 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962868/ [6962870] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:12:07 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962870/ [6962871] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:22:34 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962871/ [6962872] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:50:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962872/ [6962873] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:14:50 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962873/ [6962874] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/valgrind} time: 01:39:14 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962874/ [6962875] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:28:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962875/ [6962876] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:45:21 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962876/ [6962877] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:18:17 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962877/ [6962878] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:32:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962878/ [6962879] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 03:25:09 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962879/ [6962880] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 01:09:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962880/ [6962881] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:40:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962881/ [6962882] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:29:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962882/ [6962883] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:25:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962883/ [6962884] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:18:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962884/ [6962885] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:26:28 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962885/ [6962886] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:19:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962886/ [6962887] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:32:21 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962887/ [6962888] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962888/ [6962889] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962889/ [6962890] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:29:55 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962890/ [6962891] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:48 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962891/ [6962892] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:14:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962892/ [6962893] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962893/ [6962894] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb} time: 00:12:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962894/ [6962895] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:15:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962895/ [6962896] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:24:12 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962896/ [6962897] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962897/ [6962898] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:18:30 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962898/ [6962899] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:44:56 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962899/ [6962900] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:39:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962900/ [6962901] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:27:53 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962901/ [6962902] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:58 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962902/ [6962903] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962903/ [6962904] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:40:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962904/ [6962905] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:04 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962905/ [6962906] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 01:12:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962906/ [6962907] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:26:43 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962907/ [6962908] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962908/ [6962909] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962909/ [6962910] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962910/ [6962911] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:18:02 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962911/ [6962912] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:30:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962912/ [6962913] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:17 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962913/ [6962914] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:23:28 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962914/ [6962915] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962915/ [6962916] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962916/ [6962917] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:53:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962917/ [6962918] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:48:57 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962918/ [6962919] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 01:05:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962919/ [6962920] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/lockdep} time: 00:51:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962920/ [6962921] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 01:29:53 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962921/ [6962922] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:26:06 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962922/ [6962923] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:17:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962923/ [6962924] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:14:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962924/ [6962925] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 00:43:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962925/ [6962926] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:21:36 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962926/ [6962927] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:20:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962927/ [6962928] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 01:02:06 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962928/ [6962929] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:43:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962929/ [6962931] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:09 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962931/ [6962932] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:09 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962932/ [6962933] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962933/ [6962934] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:32:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962934/ [6962935] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:16:05 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962935/ [6962936] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:25:27 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962936/ [6962937] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 01:56:39 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962937/ [6962938] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:17:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962938/ [6962939] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962939/ [6962941] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:27:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962941/ [6962942] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:38:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962942/ [6962943] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:35:17 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962943/ [6962944] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:27:59 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962944/ [6962945] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:25:44 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962945/ [6962946] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:17:38 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962946/ [6962947] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:25:12 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962947/ [6962948] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:21:34 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962948/ [6962949] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:43:40 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962949/ [6962950] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:41:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962950/ [6962952] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962952/ [6962953] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962953/ [6962954] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:40:39 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962954/ [6962955] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:15:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962955/ [6962956] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962956/ [6962957] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962957/ [6962958] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:13:44 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962958/ [6962959] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962959/ [6962960] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 02:11:10 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962960/ [6962961] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:33:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962961/ [6962962] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962962/ [6962963] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/whitelist_health} time: 00:19:34 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962963/ [6962964] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962964/ [6962965] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:48:05 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962965/ [6962966] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:28:24 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962966/ [6962967] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962967/ [6962968] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:10:30 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962968/ [6962969] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:23:52 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962969/ [6962970] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/valgrind} time: 01:33:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962970/ [6962971] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:46:02 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962971/ [6962972] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:13:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962972/ [6962973] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:51:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962973/ [6962974] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:28:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962974/ [6962975] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:40 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962975/ [6962976] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962976/ [6962977] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:17:56 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962977/ [6962978] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:25:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962978/ [6962979] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962979/ [6962980] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:24:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962980/ [6962981] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:20:38 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962981/ [6962982] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:49:54 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962982/ [6962983] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962983/ [6962984] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:30:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962984/ [6962985] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962985/ [6962986] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:09:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962986/ [6962987] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962987/ [6962988] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 01:04:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962988/ [6962989] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:38:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962989/ [6962991] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:42:43 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962991/ [6962992] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:25:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962992/ [6962993] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962993/ [6962994] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:04 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962994/ [6962995] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:46:07 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962995/ [6962996] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962996/ [6962997] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:09:52 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962997/ [6962998] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:11:08 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962998/ [6962999] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6962999/ [6963000] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963000/ [6963001] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963001/ [6963003] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:07:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963003/ [6963004] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963004/ [6963005] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:47:04 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963005/ [6963006] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 01:01:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963006/ [6963007] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:16:53 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963007/ [6963008] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:17:35 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963008/ [6963009] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:26:34 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963009/ [6963010] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:22:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963010/ [6963011] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963011/ [6963012] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:08:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963012/ [6963013] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:45:56 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963013/ [6963014] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:31:12 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963014/ [6963015] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963015/ [6963016] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/lockdep} time: 01:02:29 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963016/ [6963017] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963017/ [6963018] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:23:28 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963018/ [6963019] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:09:35 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963019/ [6963020] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963020/ [6963021] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963021/ [6963022] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:29:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963022/ [6963023] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963023/ [6963024] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:25:49 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963024/ [6963025] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:16:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963025/ [6963026] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963026/ [6963028] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:29:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963028/ [6963029] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:22:59 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963029/ [6963030] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:34:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963030/ [6963031] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:31:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963031/ [6963032] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:32:14 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963032/ [6963033] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963033/ [6963034] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:10:16 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963034/ [6963035] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:27:10 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963035/ [6963036] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:17:57 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963036/ [6963037] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963037/ [6963038] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:55 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963038/ [6963039] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963039/ [6963040] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:12:55 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963040/ [6963041] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:01 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963041/ [6963042] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:20:26 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963042/ [6963043] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:15:21 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963043/ [6963044] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:23:11 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963044/ [6963045] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:30:56 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963045/ [6963046] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:20:35 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963046/ [6963048] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:45:17 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963048/ [6963049] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:55:41 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963049/ [6963050] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:15:28 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963050/ [6963051] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:24:45 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963051/ [6963052] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 01:31:26 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963052/ [6963053] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:26:25 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963053/ [6963054] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963054/ [6963055] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:10:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963055/ [6963056] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:43:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963056/ [6963057] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963057/ [6963059] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:19:13 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963059/ [6963060] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:22:50 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963060/ [6963061] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:12:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963061/ [6963062] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:11:51 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963062/ [6963064] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:33:33 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963064/ [6963065] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963065/ [6963066] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:22 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963066/ [6963067] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:01:00 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963067/ [6963068] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963068/ [6963069] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 03:05:20 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963069/ [6963070] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:01:37 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963070/ [6963071] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:18:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963071/ [6963072] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:37:47 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963072/ [6963073] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:23:02 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963073/ [6963074] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963074/ [6963075] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:12:41 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963075/ [6963076] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:18:28 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963076/ [6963077] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963077/ [6963078] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963078/ [6963079] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963079/ [6963080] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:28:16 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963080/ [6963081] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:10:31 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963081/ [6963082] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:18:59 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963082/ [6963083] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:25:46 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963083/ [6963084] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:12:06 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963084/ [6963085] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:41 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963085/ [6963086] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:21:15 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963086/ [6963087] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963087/ [6963088] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:11:56 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963088/ [6963089] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:17:19 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963089/ [6963090] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:33:36 info: http://pulpito.ceph.com/teuthology-2022-08-09_03:15:04-fs-quincy-distro-default-smithi/6963090/