2023-02-10T15:57:22.845 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-10T15:57:22.846 INFO:teuthology.results:subset: '111/120000' 2023-02-10T15:57:22.846 INFO:teuthology.results:seed: '2814' 2023-02-10T18:16:45.883 INFO:root:teuthology version: 0.0.1.dev37+g00690ec 2023-02-10T18:16:45.905 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-02-10T21:25:01.922 INFO:teuthology.results:Tests finished! gathering results... 2023-02-10T21:25:17.398 INFO:scrape:Found 184 jobs 2023-02-10T21:25:33.917 INFO:scrape:Found 11 distinct failure reasons 2023-02-10T21:25:33.918 INFO:scrape:Failure: Test failure: test_subvolume_snapshot_info_if_orphan_clone (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-02-10T21:25:33.918 INFO:scrape:2 jobs: ['7155602', '7155475'] 2023-02-10T21:25:33.918 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/clone}}'] 2023-02-10T21:25:33.919 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/clone}}'] 2023-02-10T21:25:33.919 INFO:scrape: 2023-02-10T21:25:33.919 INFO:scrape:Failure: "2023-02-10T17:40:23.196803+0000 mon.a (mon.0) 2382 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs degraded (PG_DEGRADED)" in cluster log 2023-02-10T21:25:33.919 INFO:scrape:2 jobs: ['7155528', '7155591'] 2023-02-10T21:25:33.919 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health'] 2023-02-10T21:25:33.919 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/exports}', 'tasks/snap-schedule}'] 2023-02-10T21:25:33.919 INFO:scrape: 2023-02-10T21:25:33.919 INFO:scrape:Failure: error during scrub thrashing: Command failed on smithi105 with status 13: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status' 2023-02-10T21:25:33.919 INFO:scrape:1 jobs: ['7155490'] 2023-02-10T21:25:33.920 INFO:scrape:suites: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'ranks/5', 'scrub/yes', 'session_timeout}', 'standby-replay', 'tasks/{0-check-counter', 'workunit/suites/dbench}', 'wsync/{no}}'] 2023-02-10T21:25:33.920 INFO:scrape: 2023-02-10T21:25:33.920 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2023-02-10T21:25:33.920 INFO:scrape:1 jobs: ['7155513'] 2023-02-10T21:25:33.920 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}'] 2023-02-10T21:25:33.920 INFO:scrape: 2023-02-10T21:25:33.920 INFO:scrape:Crash: Test failure: test_rebuild_moved_file (tasks.cephfs.test_data_scan.TestDataScan) 2023-02-10T21:25:33.920 INFO:scrape:ceph version 17.2.5-774-ge4842e6d (e4842e6def3c0c51750b118e8ec702f6124b84a2) quincy (stable) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fe19aebcb20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe19bebe493] 5: /usr/lib64/ceph/libceph-common.so.2(+0x2695ff) [0x7fe19bebe5ff] 6: (MDLog::_recovery_thread(MDSContext*)+0xb56) [0x55841e7da656] 7: (MDLog::RecoveryThread::entry()+0x15) [0x55841e490595] 8: /lib64/libpthread.so.0(+0x814a) [0x7fe19aeb214a] 9: clone() 2023-02-10T21:25:33.920 INFO:scrape:1 jobs: ['7155526'] 2023-02-10T21:25:33.921 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/data-scan}'] 2023-02-10T21:25:33.921 INFO:scrape: 2023-02-10T21:25:33.921 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4842e6def3c0c51750b118e8ec702f6124b84a2 2023-02-10T21:25:33.921 INFO:scrape:2 jobs: ['7155542', '7155609'] 2023-02-10T21:25:33.921 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2023-02-10T21:25:33.921 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}'] 2023-02-10T21:25:33.921 INFO:scrape: 2023-02-10T21:25:33.921 INFO:scrape:Failure: Test failure: test_grow_shrink (tasks.cephfs.test_failover.TestMultiFilesystems) 2023-02-10T21:25:33.922 INFO:scrape:1 jobs: ['7155599'] 2023-02-10T21:25:33.922 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_8}', 'fs/multifs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug}', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/failover}'] 2023-02-10T21:25:33.922 INFO:scrape: 2023-02-10T21:25:33.922 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4842e6def3c0c51750b118e8ec702f6124b84a2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-02-10T21:25:33.922 INFO:scrape:1 jobs: ['7155548'] 2023-02-10T21:25:33.922 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2023-02-10T21:25:33.922 INFO:scrape: 2023-02-10T21:25:33.922 INFO:scrape:Failure: "2023-02-10T17:31:19.979652+0000 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 451622.431002 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log 2023-02-10T21:25:33.922 INFO:scrape:1 jobs: ['7155551'] 2023-02-10T21:25:33.923 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/mds-full}'] 2023-02-10T21:25:33.923 INFO:scrape: 2023-02-10T21:25:33.923 INFO:scrape:Failure: "2023-02-10T18:59:06.873426+0000 mds.d (mds.0) 35 : cluster [WRN] client.4592 isn't responding to mclientcaps(revoke), ino 0x10000006032 pending pAsLsXsFscr issued pAsLsXsFscrb, sent 302.060164 seconds ago" in cluster log 2023-02-10T21:25:33.923 INFO:scrape:1 jobs: ['7155613'] 2023-02-10T21:25:33.923 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}'] 2023-02-10T21:25:33.923 INFO:scrape: 2023-02-10T21:25:33.923 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) 2023-02-10T21:25:33.923 INFO:scrape:1 jobs: ['7155625'] 2023-02-10T21:25:33.923 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/acls}'] 2023-02-10T21:25:33.924 INFO:scrape: 2023-02-10T21:25:33.934 INFO:teuthology.results:Sending results to yweinstein@radhat.com: Test Run: yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/ failed: 14 dead: 0 running: 0 waiting: 0 queued: 0 passed: 170 Fail ================================================================= [7155475] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 01:02:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155475/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155475/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e40e9f125134eaabebf1e7f04c66c5c Test failure: test_subvolume_snapshot_info_if_orphan_clone (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [7155490] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} ----------------------------------------------------------------- time: 01:49:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155490/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155490/ error during scrub thrashing: Command failed on smithi105 with status 13: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status' [7155513] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} ----------------------------------------------------------------- time: 01:23:09 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155513/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155513/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4678a370eed74457b0bdd02cb693f892 Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring) [7155526] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} ----------------------------------------------------------------- time: 00:24:20 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155526/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155526/ sentry: https://sentry.ceph.com/organizations/ceph/?query=76e85fe29fb448c09765a84f2a24e4f2 Test failure: test_rebuild_moved_file (tasks.cephfs.test_data_scan.TestDataScan) [7155528] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} ----------------------------------------------------------------- time: 00:57:22 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155528/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155528/ "2023-02-10T17:40:23.196803+0000 mon.a (mon.0) 2382 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs degraded (PG_DEGRADED)" in cluster log [7155542] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:09:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155542/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155542/ sentry: https://sentry.ceph.com/organizations/ceph/?query=847c38edbc6f4355b90c70200e126e08 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1 =e4842e6def3c0c51750b118e8ec702f6124b84a2 [7155548] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:33:45 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155548/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155548/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c026a39fb2d84673b1ad27ace38b70bd Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4842e6def3c0c51750b118e8ec702f6124b84a2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git- ceph.sh' [7155551] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} ----------------------------------------------------------------- time: 00:21:49 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155551/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155551/ "2023-02-10T17:31:19.979652+0000 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 451622.431002 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log [7155591] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:56:07 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155591/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155591/ "2023-02-10T18:31:34.246816+0000 mon.a (mon.0) 491 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [7155599] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} ----------------------------------------------------------------- time: 00:35:44 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155599/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155599/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61217d6af57b40f99ff786ae34759f64 Test failure: test_grow_shrink (tasks.cephfs.test_failover.TestMultiFilesystems) [7155602] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:53:18 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155602/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155602/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dbbddc4166e649e7ae1c51b39dcbdd80 Test failure: test_subvolume_snapshot_info_if_orphan_clone (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [7155609] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:09:25 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155609/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155609/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48ef071062014ff1b4f5e106b4349ea6 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1 =e4842e6def3c0c51750b118e8ec702f6124b84a2 [7155613] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 03:12:18 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155613/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155613/ "2023-02-10T18:59:06.873426+0000 mds.d (mds.0) 35 : cluster [WRN] client.4592 isn't responding to mclientcaps(revoke), ino 0x10000006032 pending pAsLsXsFscr issued pAsLsXsFscrb, sent 302.060164 seconds ago" in cluster log [7155625] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} ----------------------------------------------------------------- time: 00:12:27 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155625/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155625/ sentry: https://sentry.ceph.com/organizations/ceph/?query=831ef89c28ff4eefa14a969c08ff8e37 Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) Pass ================================================================= [7155443] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155443/ [7155444] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:23:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155444/ [7155445] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:30:30 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155445/ [7155446] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:30:14 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155446/ [7155447] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:25:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155447/ [7155448] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155448/ [7155449] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155449/ [7155450] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155450/ [7155451] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} time: 00:30:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155451/ [7155452] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:23:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155452/ [7155453] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:25:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155453/ [7155454] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155454/ [7155455] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:59:39 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155455/ [7155456] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:23:51 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155456/ [7155457] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155457/ [7155458] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} time: 00:12:49 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155458/ [7155459] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155459/ [7155460] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} time: 00:18:14 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155460/ [7155461] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:44 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155461/ [7155462] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155462/ [7155463] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} time: 00:14:45 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155463/ [7155464] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155464/ [7155465] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155465/ [7155466] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 02:12:21 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155466/ [7155467] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:13:49 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155467/ [7155468] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155468/ [7155469] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:20 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155469/ [7155470] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep} time: 00:21:48 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155470/ [7155471] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155471/ [7155472] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} time: 00:14:29 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155472/ [7155473] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} time: 01:00:02 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155473/ [7155474] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:06:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155474/ [7155476] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:10:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155476/ [7155477] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155477/ [7155478] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} time: 00:56:05 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155478/ [7155479] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:24:13 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155479/ [7155480] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155480/ [7155481] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:04 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155481/ [7155482] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:44:13 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155482/ [7155483] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155483/ [7155484] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:21:20 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155484/ [7155485] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:32:34 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155485/ [7155486] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree} time: 00:28:31 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155486/ [7155487] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:33 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155487/ [7155488] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair} time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155488/ [7155489] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:24:48 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155489/ [7155491] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155491/ [7155492] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155492/ [7155493] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:22:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155493/ [7155494] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush} time: 00:18:17 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155494/ [7155495] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 01:48:01 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155495/ [7155496] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155496/ [7155497] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} time: 00:13:56 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155497/ [7155498] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} time: 00:49:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155498/ [7155499] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} time: 00:20:29 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155499/ [7155500] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} time: 00:22:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155500/ [7155501] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155501/ [7155502] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 01:57:06 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155502/ [7155503] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155503/ [7155504] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:42:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155504/ [7155505] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:51:00 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155505/ [7155506] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:29:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155506/ [7155507] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155507/ [7155508] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} time: 00:38:51 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155508/ [7155509] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:46:18 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155509/ [7155510] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop} time: 00:11:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155510/ [7155511] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:22:20 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155511/ [7155512] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:22 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155512/ [7155514] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} time: 00:20:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155514/ [7155515] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 01:03:43 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155515/ [7155516] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead} time: 00:12:42 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155516/ [7155517] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} time: 00:53:10 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155517/ [7155518] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:51 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155518/ [7155519] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:03 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155519/ [7155520] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} time: 00:36:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155520/ [7155521] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:19:25 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155521/ [7155522] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} time: 00:21:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155522/ [7155523] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155523/ [7155524] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:27:23 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155524/ [7155525] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:21:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155525/ [7155527] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155527/ [7155529] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:15:14 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155529/ [7155530] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155530/ [7155531] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155531/ [7155532] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:31:36 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155532/ [7155533] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155533/ [7155534] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:20:44 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155534/ [7155535] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155535/ [7155536] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} time: 00:19:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155536/ [7155537] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155537/ [7155538] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} time: 00:59:29 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155538/ [7155539] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:49:26 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155539/ [7155540] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:30:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155540/ [7155541] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:22:19 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155541/ [7155543] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:55:56 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155543/ [7155544] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:09 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155544/ [7155545] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155545/ [7155546] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:35:30 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155546/ [7155547] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} time: 00:12:29 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155547/ [7155549] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:27:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155549/ [7155550] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep} time: 00:21:03 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155550/ [7155552] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:49:03 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155552/ [7155553] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155553/ [7155554] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155554/ [7155555] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155555/ [7155556] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155556/ [7155557] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:01 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155557/ [7155558] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:25:47 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155558/ [7155559] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:45:58 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155559/ [7155560] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} time: 00:28:28 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155560/ [7155561] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155561/ [7155562] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:15:29 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155562/ [7155563] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155563/ [7155564] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155564/ [7155565] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} time: 00:21:24 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155565/ [7155566] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:30:49 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155566/ [7155567] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:45:25 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155567/ [7155568] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:41:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155568/ [7155569] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:16 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155569/ [7155570] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:26:27 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155570/ [7155571] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:39:41 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155571/ [7155572] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} time: 00:13:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155572/ [7155573] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 01:30:16 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155573/ [7155574] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155574/ [7155575] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:56:09 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155575/ [7155576] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:26 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155576/ [7155577] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:59:30 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155577/ [7155578] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:22:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155578/ [7155579] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155579/ [7155580] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155580/ [7155581] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} time: 00:18:52 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155581/ [7155582] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:40:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155582/ [7155583] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155583/ [7155584] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} time: 00:52:52 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155584/ [7155585] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} time: 00:21:59 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155585/ [7155586] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155586/ [7155587] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:24:28 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155587/ [7155588] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} time: 00:14:38 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155588/ [7155589] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155589/ [7155590] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:27:26 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155590/ [7155592] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:34:15 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155592/ [7155593] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155593/ [7155594] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155594/ [7155595] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health} time: 00:20:33 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155595/ [7155596] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155596/ [7155597] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155597/ [7155598] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:51:43 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155598/ [7155600] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:40 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155600/ [7155601] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:01:36 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155601/ [7155603] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155603/ [7155604] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:11 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155604/ [7155605] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155605/ [7155606] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:30:57 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155606/ [7155607] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} time: 00:41:24 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155607/ [7155608] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:20:59 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155608/ [7155610] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:18:21 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155610/ [7155611] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} time: 00:13:01 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155611/ [7155612] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155612/ [7155614] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155614/ [7155615] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:13:01 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155615/ [7155616] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:29:01 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155616/ [7155617] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} time: 00:20:41 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155617/ [7155618] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:34:50 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155618/ [7155619] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind} time: 01:27:08 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155619/ [7155620] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:17:41 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155620/ [7155621] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155621/ [7155622] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:45 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155622/ [7155623] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155623/ [7155624] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:26:28 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155624/ [7155626] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:14:23 info: http://pulpito.ceph.com/yuriw-2023-02-07_20:42:45-fs-wip-yuri5-testing-2023-02-07-0850-quincy-distro-default-smithi/7155626/