2023-12-18T09:35:38.497 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-18T09:35:38.497 INFO:teuthology.results:subset: '124/1024'
2023-12-18T09:35:38.498 INFO:teuthology.results:seed: '1969'
2023-12-18T10:19:41.615 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-18T10:19:41.619 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-12-18T12:16:49.819 INFO:teuthology.results:Tests finished! gathering results...
2023-12-18T12:16:53.811 INFO:scrape:Found 53 jobs
2023-12-18T12:16:55.265 WARNING:scrape:Misunderstood line: 
2023-12-18T12:16:55.390 INFO:scrape:Found 5 distinct failure reasons
2023-12-18T12:16:55.391 INFO:scrape:Failure: "2023-12-18T10:22:50.005207+0000 mon.a (mon.0) 791 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log
2023-12-18T12:16:55.391 INFO:scrape:2 jobs: ['7495411', '7495425']
2023-12-18T12:16:55.392 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health']
2023-12-18T12:16:55.392 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/multimds_misc}', 'tasks/volumes/{overrides', 'test/clone}}']
2023-12-18T12:16:55.392 INFO:scrape:
2023-12-18T12:16:55.392 INFO:scrape:Failure: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)
2023-12-18T12:16:55.392 INFO:scrape:1 jobs: ['7495424']
2023-12-18T12:16:55.392 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/ubuntu_latest', 'fs/shell/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs-shell}']
2023-12-18T12:16:55.392 INFO:scrape:
2023-12-18T12:16:55.392 INFO:scrape:Failure: Command failed on smithi097 with status 2: 'sudo dpkg -i /tmp/linux-image.deb'
2023-12-18T12:16:55.392 INFO:scrape:1 jobs: ['7495388']
2023-12-18T12:16:55.393 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/admin}']
2023-12-18T12:16:55.393 INFO:scrape:
2023-12-18T12:16:55.393 INFO:scrape:Valgrind: mon (InvalidRead, Leak_PossiblyLost, Leak_StillReachable)
2023-12-18T12:16:55.393 INFO:scrape:1 jobs: ['7495396']
2023-12-18T12:16:55.393 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'ranks/5', 'session_timeout}', 'tasks/fsstress', 'validater/valgrind}']
2023-12-18T12:16:55.393 INFO:scrape:
2023-12-18T12:16:55.393 INFO:scrape:Failure: Command failed on smithi043 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8688b2517b6002b804a4844986e6ed5ce973d99f pull'
2023-12-18T12:16:55.393 INFO:scrape:1 jobs: ['7495384']
2023-12-18T12:16:55.393 INFO:scrape:suites: ['fs/nfs/{cluster/{1-node}', 'overrides/ignorelist_health', 'supported-random-distros$/{ubuntu_latest}', 'tasks/nfs}']
2023-12-18T12:16:55.394 INFO:scrape:
2023-12-18T12:16:55.396 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi
=================================================================
info:    https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  47


Fail
=================================================================
[7495384]  fs/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs}
-----------------------------------------------------------------
time:   00:10:45
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495384/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495384/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5675c46f5f37464a9ce8eca37fb7ab73

    Command failed on smithi043 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay-quay-
    quay.apps.os.sepia.ceph.com/ceph-
    ci/ceph:8688b2517b6002b804a4844986e6ed5ce973d99f pull'

[7495388]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495388/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495388/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bfedb35eaf63467b805ad1bafea9cf55

    Command failed on smithi097 with status 2: 'sudo dpkg -i /tmp/linux-
    image.deb'

[7495396]  fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:31:23
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495396/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495396/
sentry: https://sentry.ceph.com/organizations/ceph/?query=891bd5560f8b485a80f2efe572c6cf46

    saw valgrind issues

[7495411]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
-----------------------------------------------------------------
time:   00:25:43
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495411/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495411/

    "2023-12-18T10:22:50.005207+0000 mon.a (mon.0) 791 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log

[7495424]  fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
-----------------------------------------------------------------
time:   00:14:44
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495424/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495424/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fc49295878974e0aaafbd271d1bc4af8

    Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)

[7495425]  fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
-----------------------------------------------------------------
time:   01:00:01
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495425/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495425/

    "2023-12-18T10:53:18.488895+0000 mon.a (mon.0) 2913 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in
    cluster log



Pass
=================================================================
[7495381] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_20.04} tasks/mirror}
time:   02:17:38
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495381/

[7495382] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{ubuntu_20.04} workloads/cephfs-mirror-ha-workunit}
time:   00:39:17
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495382/

[7495383] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:04:13
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495383/

[7495385] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/fstop}
time:   00:16:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495385/

[7495386] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:21:14
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495386/

[7495387] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/lockdep}
time:   00:50:34
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495387/

[7495389] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:20:48
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495389/

[7495390] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:19:08
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495390/

[7495391] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}}
time:   00:20:39
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495391/

[7495392] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:21:26
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495392/

[7495393] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:21:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495393/

[7495394] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:19:48
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495394/

[7495395] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}
time:   00:25:19
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495395/

[7495397] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:21:33
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495397/

[7495398] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:25:37
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495398/

[7495399] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:16:47
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495399/

[7495400] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:21:48
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495400/

[7495401] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:53:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495401/

[7495402] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:25:39
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495402/

[7495403] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:23:37
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495403/

[7495404] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:23:05
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495404/

[7495405] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:19:12
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495405/

[7495406] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-ffsb}
time:   02:01:09
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495406/

[7495407] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:35:22
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495407/

[7495408] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:04:27
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495408/

[7495409] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:37:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495409/

[7495410] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:22:37
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495410/

[7495412] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}
time:   00:32:48
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495412/

[7495413] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:21:03
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495413/

[7495414] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-iozone}
time:   02:20:15
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495414/

[7495415] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_20.04 mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
time:   00:52:28
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495415/

[7495416] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:24:46
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495416/

[7495417] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:43:08
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495417/

[7495418] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:16:21
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495418/

[7495419] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
time:   01:51:32
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495419/

[7495420] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}
time:   00:40:17
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495420/

[7495421] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python}
time:   00:19:54
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495421/

[7495422] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:21:34
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495422/

[7495423] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:25:41
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495423/

[7495426] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   00:27:38
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495426/

[7495427] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:16:57
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495427/

[7495428] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:21:30
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495428/

[7495429] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:34:18
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495429/

[7495430] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/lockdep}
time:   00:51:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495430/

[7495431] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
time:   00:21:36
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495431/

[7495432] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:19:31
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495432/

[7495433] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}}
time:   00:19:56
info:   https://pulpito.ceph.com/vshankar-2023-12-18_09:35:32-fs-wip-vshankar-testing1-2023-12-18-1207-reef-testing-default-smithi/7495433/