2023-03-01T07:08:10.547 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8 2023-03-01T07:08:10.548 INFO:teuthology.results:subset: '64/512' 2023-03-01T07:08:10.548 INFO:teuthology.results:seed: '6101' 2023-03-01T08:03:26.585 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8 2023-03-01T08:03:26.614 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-03-01T19:54:19.379 INFO:teuthology.results:Tests finished! gathering results... 2023-03-01T19:54:21.417 INFO:scrape:Found 22 jobs 2023-03-01T19:54:27.460 INFO:scrape:Found 6 distinct failure reasons 2023-03-01T19:54:27.461 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f9d812a56231a14fafcdfb339f87d3d9a9e6e55f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2023-03-01T19:54:27.461 INFO:scrape:2 jobs: ['7190062', '7190081'] 2023-03-01T19:54:27.461 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '4-flush/yes', '5-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'session_timeout}', 'standby-replay'] 2023-03-01T19:54:27.462 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '2-scrub/yes', '3-snaps/no', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/1', 'ranks/multi/{export-check', 'replication/default}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}'] 2023-03-01T19:54:27.462 INFO:scrape: 2023-03-01T19:54:27.462 INFO:scrape:Dead: b'2023-03-01T19:24:09.437 DEBUG:teuthology.exit:Finished running handlers' 2023-03-01T19:54:27.463 INFO:scrape:1 jobs: ['7190065'] 2023-03-01T19:54:27.463 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'n/3', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-no-extra-options}'] 2023-03-01T19:54:27.463 INFO:scrape: 2023-03-01T19:54:27.464 INFO:scrape:Failure: error during scrub thrashing: reached maximum tries (30) after waiting for 900 seconds 2023-03-01T19:54:27.464 INFO:scrape:2 jobs: ['7190072', '7190076'] 2023-03-01T19:54:27.464 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay'] 2023-03-01T19:54:27.464 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/kernel_untar_build}}', '5-workunit/suites/blogbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{no-subvolume}', 'tasks/{0-subvolume/{with-namespace-isolated}', 'wsync/no}'] 2023-03-01T19:54:27.465 INFO:scrape: 2023-03-01T19:54:27.465 INFO:scrape:Failure: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-03-01T19:54:27.465 INFO:scrape:1 jobs: ['7190067'] 2023-03-01T19:54:27.466 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/data-scan}'] 2023-03-01T19:54:27.466 INFO:scrape: 2023-03-01T19:54:27.466 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2023-03-01T19:54:27.466 INFO:scrape:1 jobs: ['7190064'] 2023-03-01T19:54:27.467 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}'] 2023-03-01T19:54:27.467 INFO:scrape: 2023-03-01T19:54:27.467 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/suites/fsstress.sh 2023-03-01T19:54:27.468 INFO:scrape:1 jobs: ['7190078'] 2023-03-01T19:54:27.468 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/fsstress}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/5', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'wsync/no}'] 2023-03-01T19:54:27.468 INFO:scrape: 2023-03-01T19:54:27.470 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/ failed: 7 dead: 1 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [7190062] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:25:35 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190062/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190062/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f9339a89bdc48eda8ca78f1465d9166 Command failed (workunit test suites/pjd.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f9d812a56231a14fafcdfb339f87d3d9a9e6e55f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [7190064] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} ----------------------------------------------------------------- time: 01:38:14 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190064/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190064/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c1e4797af49492c8e97a797d9d2e034 Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) [7190067] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} ----------------------------------------------------------------- time: 00:20:41 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190067/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190067/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3850482e72864f92b598aeaa86cf251a Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) [7190072] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 02:43:55 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190072/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190072/ error during scrub thrashing: reached maximum tries (30) after waiting for 900 seconds [7190076] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:57:26 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190076/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190076/ error during scrub thrashing: rank damage found: {'backtrace'} [7190078] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 06:35:08 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190078/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190078/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37f36e74580d42799fb0923f558a3857 Command failed (workunit test suites/fsstress.sh) on smithi031 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f9d812a56231a14fafcdfb339f87d3d9a9e6e55f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [7190081] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:41:29 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190081/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190081/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1589c2725bf44b6dba1693ffbd405fcb Command failed (workunit test suites/pjd.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f9d812a56231a14fafcdfb339f87d3d9a9e6e55f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' Dead ================================================================= [7190065] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190065/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190065/ hit max job timeout Pass ================================================================= [7190060] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:47:14 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190060/ [7190061] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} time: 00:21:01 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190061/ [7190063] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 02:22:50 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190063/ [7190066] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} time: 00:31:40 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190066/ [7190068] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} time: 00:24:19 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190068/ [7190069] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:19:58 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190069/ [7190070] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} time: 00:28:51 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190070/ [7190071] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} time: 00:48:03 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190071/ [7190073] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} time: 00:44:27 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190073/ [7190074] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:35:04 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190074/ [7190075] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:49:20 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190075/ [7190077] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:19:14 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190077/ [7190079] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:42:35 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190079/ [7190080] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} time: 00:20:56 info: http://pulpito.ceph.com/vshankar-2023-03-01_07:02:38-fs-main-testing-default-smithi/7190080/