2021-11-17T04:34:18.984 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-17T04:34:18.985 INFO:teuthology.results:subset: '3/32' 2021-11-17T04:34:18.986 INFO:teuthology.results:seed: '109' 2021-11-17T10:39:36.169 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-17T10:39:36.206 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-17T14:18:55.754 INFO:teuthology.results:Tests finished! gathering results... 2021-11-17T14:19:19.657 INFO:scrape:Found 267 jobs 2021-11-17T14:19:36.781 INFO:scrape:Found 8 distinct failure reasons 2021-11-17T14:19:36.783 INFO:scrape:Failure: "2021-11-17T06:42:38.033176+0000 mon.a (mon.0) 384 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log 2021-11-17T14:19:36.783 INFO:scrape:6 jobs: ['6507972', '6507997', '6507963', '6507903', '6508112', '6507947'] 2021-11-17T14:19:36.784 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.784 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-comp', 'objectstore/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'overrides/{no_client_pidfile', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/client-recovery}', 'tasks/snap-schedule}', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/ffsb}', 'workunit/suites/fsync-tester}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}'] 2021-11-17T14:19:36.785 INFO:scrape: 2021-11-17T14:19:36.785 INFO:scrape:Failure: Command failed on smithi101 with status 5: 'sudo systemctl stop ceph-dd19003a-4762-11ec-8c2c-001a4aab830c@mon.smithi101' 2021-11-17T14:19:36.786 INFO:scrape:1 jobs: ['6507878'] 2021-11-17T14:19:36.786 INFO:scrape:suites: ['1-mount', '2-workload/dbench', 'distro/centos_latest}', 'fs/cephadm/multivolume/{0-start'] 2021-11-17T14:19:36.787 INFO:scrape: 2021-11-17T14:19:36.787 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh' 2021-11-17T14:19:36.788 INFO:scrape:15 jobs: ['6507981', '6508124', '6508037', '6507948', '6507910', '6508004', '6507976', '6507922', '6508093', '6508008', '6507959', '6508068', '6507896', '6508053', '6508098'] 2021-11-17T14:19:36.788 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.789 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_snaptests}}', '2-workunit/fs/snaps}}', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/snaps/{begin', 'fs/thrash/multifs/{begin', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'overrides/{whitelist_health', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/workunit/snaps}', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.789 INFO:scrape: 2021-11-17T14:19:36.790 INFO:scrape:Failure: Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays) 2021-11-17T14:19:36.790 INFO:scrape:1 jobs: ['6508123'] 2021-11-17T14:19:36.791 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/strays}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.791 INFO:scrape: 2021-11-17T14:19:36.792 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2021-11-17T14:19:36.792 INFO:scrape:3 jobs: ['6507970', '6507874', '6507964'] 2021-11-17T14:19:36.793 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag', 'races', 'session_timeout', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.793 INFO:scrape:suites union: ['2-workunit/fs/trivial_sync}}', '2-workunit/suites/pjd}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{centos_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'races', 'ranks/1', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.794 INFO:scrape: 2021-11-17T14:19:36.794 INFO:scrape:Crash: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi029 with status 23: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2021-11-17T14:19:36.795 INFO:scrape:ceph version 17.0.0-9064-gdea6ee7c (dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f14a5ba0b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7f14a7e6f944] 5: /usr/lib64/ceph/libceph-common.so.2(+0x281b07) [0x7f14a7e6fb07] 6: (Monitor::~Monitor()+0xb35) [0x56491011d955] 7: (Monitor::~Monitor()+0xd) [0x56491011d9ad] 8: main() 9: __libc_start_main() 10: _start() 2021-11-17T14:19:36.795 INFO:scrape:2 jobs: ['6508025', '6507979'] 2021-11-17T14:19:36.796 INFO:scrape:suites intersection: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'mds', 'mon', 'ms_mode/{secure}', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}'] 2021-11-17T14:19:36.796 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'ranks/3', 'ranks/5', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'wsync/{no}}', 'wsync/{yes}}'] 2021-11-17T14:19:36.797 INFO:scrape: 2021-11-17T14:19:36.797 INFO:scrape:Dead: b'2021-11-17T14:17:03.611 DEBUG:teuthology.exit:Finished running handlers' 2021-11-17T14:19:36.798 INFO:scrape:1 jobs: ['6507942'] 2021-11-17T14:19:36.798 INFO:scrape:suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}', 'wsync/{no}}'] 2021-11-17T14:19:36.799 INFO:scrape: 2021-11-17T14:19:36.799 INFO:scrape:Failure: 2021-11-17T14:19:36.800 INFO:scrape:4 jobs: ['6508100', '6507912', '6508055', '6507983'] 2021-11-17T14:19:36.800 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/{pacific', 'v16.2.4}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.801 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/{pacific', 'v16.2.4}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-17T14:19:36.801 INFO:scrape: 2021-11-17T14:19:36.813 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/ failed: 32 dead: 1 running: 0 waiting: 0 queued: 0 passed: 234 Fail ================================================================= [6507874] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:19:23 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507874/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2256d01a018e4d44bd755d95da75f631 reached maximum tries (90) after waiting for 540 seconds [6507878] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} ----------------------------------------------------------------- time: 00:08:29 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507878/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507878/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7771b9cf55ad4f65ae618ad433429254 Command failed on smithi101 with status 5: 'sudo systemctl stop ceph- dd19003a-4762-11ec-8c2c-001a4aab830c@mon.smithi101' [6507896] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:25:08 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507896/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507896/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aff2ebe4c5994f2fa00c3eef3ad85533 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507903] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:20:11 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507903/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507903/ "2021-11-17T05:27:02.243256+0000 mon.a (mon.0) 314 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6507910] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:23:10 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507910/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=525c6c5b76874fc09aef1a4007519eef Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507912] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/{pacific v16.2.4} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:09:19 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507912/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507912/ sentry: https://sentry.ceph.com/organizations/ceph/?query=22c060d53ed047a49ff4542ac382ddef [6507922] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:22:05 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507922/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507922/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3e81aa49d15f48c9847834cb0c76b476 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi140 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507947] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} ----------------------------------------------------------------- time: 01:22:44 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507947/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507947/ "2021-11-17T06:16:56.018137+0000 mon.a (mon.0) 175 : cluster [WRN] Health check failed: Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6507948] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:21:13 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507948/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507948/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aae3c2837672427fae259395a3ffd270 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507959] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:19:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507959/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507959/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ab23024d6be1403ab56c8464a23fc85d Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi114 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507963] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} ----------------------------------------------------------------- time: 00:13:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507963/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507963/ "2021-11-17T06:29:45.092023+0000 mon.a (mon.0) 158 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [6507964] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:19:22 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507964/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507964/ sentry: https://sentry.ceph.com/organizations/ceph/?query=694ee2cfab2b4f6081697ad9006a85b6 reached maximum tries (90) after waiting for 540 seconds [6507970] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:22:10 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507970/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507970/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7f61d49552a4870bbb08435b79a40b9 reached maximum tries (90) after waiting for 540 seconds [6507972] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:14:18 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507972/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507972/ "2021-11-17T06:42:38.033176+0000 mon.a (mon.0) 384 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [6507976] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:23:44 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507976/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507976/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0dd49a60b2094db2995b49ab419a822d Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507979] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} ----------------------------------------------------------------- time: 03:32:47 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507979/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507979/ sentry: https://sentry.ceph.com/organizations/ceph/?query=23e43080870b4ac08d5b8791f9d1b4b0 Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi008 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/misc/multiple_rsync.sh' [6507981] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:26:14 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507981/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507981/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b3afdc5b8c54587bece21fb3e27c7cd Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6507983] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/{pacific v16.2.4} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:09:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507983/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507983/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a786283be2947c89fc1c6ef2976c848 [6507997] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:31:03 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507997/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507997/ "2021-11-17T07:14:46.288544+0000 mon.a (mon.0) 508 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive (PG_AVAILABILITY)" in cluster log [6508004] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:19:53 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508004/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508004/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29764c9f3fd840e7ac01498ed0d49caf Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi172 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508008] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:41:35 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508008/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508008/ sentry: https://sentry.ceph.com/organizations/ceph/?query=60fda62ce6f347f386dfff1ec6780e50 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi118 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508025] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 01:05:37 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508025/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a713f8fd22f4f15a72e37df26d50517 Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi029 with status 23: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/misc/multiple_rsync.sh' [6508037] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:24:33 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508037/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508037/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fe8c7d48c10b498695b9fd94742aa186 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi087 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508053] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:18:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508053/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508053/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e02a85542bc40669f185f799a13afeb Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi085 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508055] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/{pacific v16.2.4} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:09:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508055/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508055/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38aca79e8be64f288b5d39b221be8210 [6508068] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:33:08 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508068/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508068/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1981930af677426da01f3643120c181f Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508093] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:33:51 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508093/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508093/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e9e3c76e259c427cbf90c1e9126b024f Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508098] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:20:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508098/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508098/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8d0056e3d2f44e4c8ee8928bb3dce987 Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6508100] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/{pacific v16.2.4} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508100/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508100/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e45b8f60a61a47bab33d663d8eb7441b [6508112] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:16:58 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508112/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508112/ "2021-11-17T10:37:52.481759+0000 mon.a (mon.0) 438 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6508123] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} ----------------------------------------------------------------- time: 00:27:40 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508123/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508123/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b1505e6cebf64268b6c560bab676808c Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays) [6508124] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:22:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508124/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508124/ sentry: https://sentry.ceph.com/organizations/ceph/?query=88d3e35afcf541ffb3df85548e75837c Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dea6ee7c0d8a1db3fa9f76b99e3c35b56d3eacba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' Dead ================================================================= [6507942] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507942/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507942/ Pass ================================================================= [6507872] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:18:03 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507872/ [6507873] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} time: 00:11:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507873/ [6507875] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:15:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507875/ [6507876] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:18:02 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507876/ [6507877] fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137} time: 00:11:49 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507877/ [6507879] fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full} time: 00:15:31 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507879/ [6507880] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:11:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507880/ [6507881] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} time: 01:54:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507881/ [6507882] fs/mirror-ha/{begin cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:44 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507882/ [6507883] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:33:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507883/ [6507884] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:20:01 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507884/ [6507885] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:24:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507885/ [6507886] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:13:14 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507886/ [6507887] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:34:22 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507887/ [6507888] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:15:54 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507888/ [6507889] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{centos_8.stream} tasks/fstop} time: 00:11:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507889/ [6507890] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:47 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507890/ [6507891] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 02:04:41 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507891/ [6507892] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:52:01 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507892/ [6507893] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:15:49 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507893/ [6507894] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:21:54 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507894/ [6507895] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:32:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507895/ [6507897] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:42:55 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507897/ [6507898] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:16:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507898/ [6507899] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:04:13 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507899/ [6507900] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:21:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507900/ [6507901] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:26:39 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507901/ [6507902] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:01:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507902/ [6507904] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 02:32:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507904/ [6507905] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:34 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507905/ [6507906] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:15:39 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507906/ [6507907] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:21:03 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507907/ [6507908] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:16:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507908/ [6507909] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:16:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507909/ [6507911] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 01:08:10 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507911/ [6507913] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:39:05 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507913/ [6507914] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:17:33 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507914/ [6507915] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:30:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507915/ [6507916] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:28:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507916/ [6507917] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:16:16 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507917/ [6507918] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 01:49:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507918/ [6507919] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:24:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507919/ [6507920] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:58 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507920/ [6507921] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:45 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507921/ [6507923] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:19:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507923/ [6507924] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:35:31 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507924/ [6507925] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:47:33 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507925/ [6507926] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:15:34 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507926/ [6507927] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:13:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507927/ [6507928] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:16:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507928/ [6507929] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:16:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507929/ [6507930] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:35:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507930/ [6507931] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:10:02 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507931/ [6507932] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:52:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507932/ [6507933] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} time: 00:16:32 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507933/ [6507934] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:14:25 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507934/ [6507935] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:15:39 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507935/ [6507936] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:13:02 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507936/ [6507937] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 00:47:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507937/ [6507938] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:10:24 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507938/ [6507939] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:15:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507939/ [6507940] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 01:43:18 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507940/ [6507941] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:13:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507941/ [6507943] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:17:59 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507943/ [6507944] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 01:28:53 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507944/ [6507945] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:17:56 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507945/ [6507946] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:49:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507946/ [6507949] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:16:44 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507949/ [6507950] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:13:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507950/ [6507951] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507951/ [6507952] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8.stream} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb} time: 00:12:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507952/ [6507953] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:31 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507953/ [6507954] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:16:51 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507954/ [6507955] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:53:08 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507955/ [6507956] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:26:37 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507956/ [6507957] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:15:56 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507957/ [6507958] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:21:23 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507958/ [6507960] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:47:30 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507960/ [6507961] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:19:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507961/ [6507962] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:18:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507962/ [6507965] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:23:23 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507965/ [6507966] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:14:45 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507966/ [6507967] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:28:35 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507967/ [6507968] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:29:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507968/ [6507969] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:20:32 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507969/ [6507971] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:13:53 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507971/ [6507973] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:30:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507973/ [6507974] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:19:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507974/ [6507975] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:21:45 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507975/ [6507977] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} time: 00:50:13 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507977/ [6507978] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:13:16 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507978/ [6507980] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:12:34 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507980/ [6507982] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:29:56 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507982/ [6507984] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:31:31 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507984/ [6507985] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:29:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507985/ [6507986] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:18:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507986/ [6507987] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507987/ [6507988] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:28 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507988/ [6507989] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:49:50 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507989/ [6507990] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 00:43:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507990/ [6507991] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 00:55:21 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507991/ [6507992] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:11:13 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507992/ [6507993] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:14:47 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507993/ [6507994] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:17:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507994/ [6507995] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:16:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507995/ [6507996] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:17:28 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507996/ [6507998] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:15:05 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507998/ [6507999] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:20:48 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6507999/ [6508000] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:17:49 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508000/ [6508001] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:21:36 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508001/ [6508002] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:24:40 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508002/ [6508003] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:25:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508003/ [6508005] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:18:02 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508005/ [6508006] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:23:03 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508006/ [6508007] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:21:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508007/ [6508009] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:45:53 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508009/ [6508010] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:23:26 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508010/ [6508011] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:06:00 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508011/ [6508012] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:14:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508012/ [6508013] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:14:55 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508013/ [6508014] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:43:29 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508014/ [6508015] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:21:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508015/ [6508016] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:22 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508016/ [6508017] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}} time: 00:17:59 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508017/ [6508018] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:31:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508018/ [6508019] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:19:05 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508019/ [6508020] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:45:11 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508020/ [6508021] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:26 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508021/ [6508022] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/lockdep} time: 00:22:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508022/ [6508023] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:16:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508023/ [6508024] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:25 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508024/ [6508026] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:17:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508026/ [6508027] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:18:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508027/ [6508028] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:27:58 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508028/ [6508029] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:28:14 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508029/ [6508030] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:21:01 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508030/ [6508031] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:18:34 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508031/ [6508032] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:31:55 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508032/ [6508033] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:12:54 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508033/ [6508034] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:18:17 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508034/ [6508035] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 01:00:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508035/ [6508036] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:17:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508036/ [6508038] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:13:37 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508038/ [6508039] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:18:41 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508039/ [6508040] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:58:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508040/ [6508041] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:13:56 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508041/ [6508042] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:21:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508042/ [6508043] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:20:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508043/ [6508044] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:18:48 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508044/ [6508045] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:23:14 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508045/ [6508046] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:15:25 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508046/ [6508047] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:21:59 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508047/ [6508048] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:26:22 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508048/ [6508049] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:23:06 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508049/ [6508050] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 01:04:26 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508050/ [6508051] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:27:21 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508051/ [6508052] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:23:19 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508052/ [6508054] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:20:14 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508054/ [6508056] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:48:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508056/ [6508057] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:18:50 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508057/ [6508058] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:15:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508058/ [6508059] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:25:24 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508059/ [6508060] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:19:39 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508060/ [6508061] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:36:28 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508061/ [6508062] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:10:18 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508062/ [6508063] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:24:30 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508063/ [6508064] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:13:36 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508064/ [6508065] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/valgrind} time: 00:51:29 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508065/ [6508066] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:13:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508066/ [6508067] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 00:40:11 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508067/ [6508069] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:15:51 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508069/ [6508070] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:38:33 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508070/ [6508071] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:06:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508071/ [6508072] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:36:32 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508072/ [6508073] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:14:53 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508073/ [6508074] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:24:30 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508074/ [6508075] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:08:47 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508075/ [6508076] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:50:25 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508076/ [6508077] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:20:51 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508077/ [6508078] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:28 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508078/ [6508079] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:13:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508079/ [6508080] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:17:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508080/ [6508081] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:20:21 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508081/ [6508082] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:37 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508082/ [6508083] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:14:13 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508083/ [6508084] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:23 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508084/ [6508085] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:18:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508085/ [6508086] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:11:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508086/ [6508087] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:18:49 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508087/ [6508088] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:26:57 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508088/ [6508089] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:16:28 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508089/ [6508090] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:54:42 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508090/ [6508091] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} time: 00:11:19 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508091/ [6508092] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:25:41 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508092/ [6508094] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:39:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508094/ [6508095] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:22:16 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508095/ [6508096] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 01:14:31 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508096/ [6508097] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:14:29 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508097/ [6508099] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:44:07 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508099/ [6508101] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:33:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508101/ [6508102] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:12:06 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508102/ [6508103] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:19:20 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508103/ [6508104] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:15:25 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508104/ [6508105] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:44:51 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508105/ [6508106] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:12:08 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508106/ [6508107] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:53:36 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508107/ [6508108] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} time: 00:17:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508108/ [6508109] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:16:16 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508109/ [6508110] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:15:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508110/ [6508111] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:37:36 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508111/ [6508113] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:16:55 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508113/ [6508114] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:39:08 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508114/ [6508115] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:22:19 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508115/ [6508116] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:15:59 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508116/ [6508117] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:39 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508117/ [6508118] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:05 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508118/ [6508119] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:31:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508119/ [6508120] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 01:44:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508120/ [6508121] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:20:33 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508121/ [6508122] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 01:23:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508122/ [6508125] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:15:04 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508125/ [6508126] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:24:15 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508126/ [6508127] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8.stream} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:15:38 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508127/ [6508128] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:00:09 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508128/ [6508129] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:18:46 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508129/ [6508130] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:26:12 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508130/ [6508131] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:19:02 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508131/ [6508132] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:15:11 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508132/ [6508133] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:12:43 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508133/ [6508134] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:12:32 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508134/ [6508135] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:22:34 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508135/ [6508136] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:26:41 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508136/ [6508137] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:32:11 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508137/ [6508138] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:15:21 info: http://pulpito.ceph.com/pdonnell-2021-11-16_22:20:34-fs-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6508138/