2022-12-13T06:20:49.088 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-13T06:20:49.089 INFO:teuthology.results:subset: '111/120000' 2022-12-13T06:20:49.089 INFO:teuthology.results:seed: '9841' 2022-12-13T06:33:38.676 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-13T06:33:38.734 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-12-13T18:34:37.252 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2022-12-13T18:34:38.845 INFO:scrape:Found 16 jobs 2022-12-13T18:34:44.439 INFO:scrape:Found 7 distinct failure reasons 2022-12-13T18:34:44.440 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi138 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aa21121aa9438156e768ae1ae733b4048246362 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' 2022-12-13T18:34:44.440 INFO:scrape:3 jobs: ['7114558', '7114570', '7114573'] 2022-12-13T18:34:44.441 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2022-12-13T18:34:44.441 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2022-12-13T18:34:44.441 INFO:scrape: 2022-12-13T18:34:44.442 INFO:scrape:Failure: error during scrub thrashing: reached maximum tries (30) after waiting for 900 seconds 2022-12-13T18:34:44.442 INFO:scrape:2 jobs: ['7114571', '7114563'] 2022-12-13T18:34:44.443 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay'] 2022-12-13T18:34:44.443 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/postgres}}', '5-workunit/suites/ffsb}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'n/3', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{no-subvolume}', 'tasks/{0-subvolume/{with-quota}'] 2022-12-13T18:34:44.443 INFO:scrape: 2022-12-13T18:34:44.444 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/suites/fsstress.sh 2022-12-13T18:34:44.444 INFO:scrape:1 jobs: ['7114564'] 2022-12-13T18:34:44.445 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/suites/fsstress}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/5', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/multi/{export-check', 'replication/always}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated}', 'wsync/no}'] 2022-12-13T18:34:44.445 INFO:scrape: 2022-12-13T18:34:44.445 INFO:scrape:Failure: Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2022-12-13T18:34:44.446 INFO:scrape:1 jobs: ['7114565'] 2022-12-13T18:34:44.446 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/client-limits}'] 2022-12-13T18:34:44.446 INFO:scrape: 2022-12-13T18:34:44.447 INFO:scrape:Failure: Test failure: test_hardlink_reintegration (tasks.cephfs.test_strays.TestStrays) 2022-12-13T18:34:44.447 INFO:scrape:1 jobs: ['7114562'] 2022-12-13T18:34:44.447 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/strays}'] 2022-12-13T18:34:44.448 INFO:scrape: 2022-12-13T18:34:44.448 INFO:scrape:Dead: b'2022-12-13T18:31:07.446 DEBUG:teuthology.exit:Finished running handlers' 2022-12-13T18:34:44.448 INFO:scrape:ceph version 18.0.0-1335-g3aa21121 (3aa21121aa9438156e768ae1ae733b4048246362) reef (dev) 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f7d6ee12ce0] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x55b8715635e1] 5: (KernelDevice::_aio_thread()+0xeff) [0x55b8721fd5ff] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x55b872200221] 7: /lib64/libpthread.so.0(+0x81cf) [0x7f7d6ee081cf] 8: clone() 2022-12-13T18:34:44.449 INFO:scrape:1 jobs: ['7114567'] 2022-12-13T18:34:44.449 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2022-12-13T18:34:44.450 INFO:scrape: 2022-12-13T18:34:44.450 INFO:scrape:Failure: "1670914371.540212 mon.a (mon.0) 442 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log 2022-12-13T18:34:44.450 INFO:scrape:1 jobs: ['7114561'] 2022-12-13T18:34:44.451 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}'] 2022-12-13T18:34:44.451 INFO:scrape: 2022-12-13T18:34:44.453 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/ failed: 9 dead: 0 running: 1 waiting: 0 queued: 0 passed: 6 Fail ================================================================= [7114558] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:28:33 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114558/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114558/ sentry: https://sentry.ceph.com/organizations/ceph/?query=27ac27c17efd43b0b2394ea4bb454261 Command failed (workunit test kernel_untar_build.sh) on smithi138 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aa21121aa9438156e768ae1ae733b4048246362 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' [7114561] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:57:57 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114561/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114561/ "1670914371.540212 mon.a (mon.0) 442 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log [7114562] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} ----------------------------------------------------------------- time: 00:21:45 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114562/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114562/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20f4db6ebdff4862b00e7c626dc1f70f Test failure: test_hardlink_reintegration (tasks.cephfs.test_strays.TestStrays) [7114563] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:16:41 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114563/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114563/ reached maximum tries (50) after waiting for 300 seconds [7114564] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 06:44:45 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114564/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114564/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a4323229b6d4319b4611e4c2dd13fd7 Command failed (workunit test suites/fsstress.sh) on smithi079 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aa21121aa9438156e768ae1ae733b4048246362 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [7114565] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} ----------------------------------------------------------------- time: 00:20:49 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114565/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114565/ sentry: https://sentry.ceph.com/organizations/ceph/?query=349297c6d0b143459837dc47a9e2214c Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) [7114570] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:28:10 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114570/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114570/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7697084909234eabb7739814e163bd6a Command failed (workunit test kernel_untar_build.sh) on smithi132 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aa21121aa9438156e768ae1ae733b4048246362 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' [7114571] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} ----------------------------------------------------------------- time: 01:25:06 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114571/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114571/ error during scrub thrashing: reached maximum tries (30) after waiting for 900 seconds [7114573] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:27:21 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114573/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114573/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1e72d45e4db348cfa7253959d496cd64 Command failed (workunit test kernel_untar_build.sh) on smithi122 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aa21121aa9438156e768ae1ae733b4048246362 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' Running ================================================================= [7114567] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114567/ Pass ================================================================= [7114559] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:56:15 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114559/ [7114560] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 02:11:22 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114560/ [7114566] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} time: 00:25:27 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114566/ [7114568] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} time: 00:38:06 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114568/ [7114569] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} time: 00:19:55 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114569/ [7114572] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} time: 00:44:28 info: http://pulpito.ceph.com/vshankar-2022-12-13_05:22:50-fs-wip-vshankar-testing-20221209.043803-testing-default-smithi/7114572/