2022-03-08T07:33:12.128 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-08T07:33:12.129 INFO:teuthology.results:subset: '3/32' 2022-03-08T07:33:12.130 INFO:teuthology.results:seed: '7053' 2022-03-08T11:44:45.781 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-08T11:44:45.815 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-08T13:22:54.991 INFO:teuthology.results:Tests finished! gathering results... 2022-03-08T13:23:21.865 INFO:scrape:Found 328 jobs 2022-03-08T13:23:21.866 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726735/teuthology.log 2022-03-08T13:23:35.839 INFO:scrape:Found 12 distinct failure reasons 2022-03-08T13:23:35.840 INFO:scrape:Dead: None 2022-03-08T13:23:35.841 INFO:scrape:1 jobs: ['6726735'] 2022-03-08T13:23:35.841 INFO:scrape: 2022-03-08T13:23:35.841 INFO:scrape:Crash: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi070 with status 23: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=56eeb74f63f2a50b371e0c8c7cafef6c7dc248bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2022-03-08T13:23:35.842 INFO:scrape:ceph version 16.2.7-545-g37272975 (372729756c2ba42025d263fce78fb2410486f54f) pacific (stable) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f365dffbb20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f36602bcca9] 5: /usr/lib64/ceph/libceph-common.so.2(+0x276e72) [0x7f36602bce72] 6: (Monitor::~Monitor()+0xe40) [0x55f121a39ad0] 7: (Monitor::~Monitor()+0xd) [0x55f121a39b2d] 8: main() 9: __libc_start_main() 10: _start() 2022-03-08T13:23:35.842 INFO:scrape:2 jobs: ['6726643', '6726786'] 2022-03-08T13:23:35.843 INFO:scrape:suites intersection: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.844 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'ranks/3', 'ranks/5', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'workunit/suites/iogen}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-03-08T13:23:35.844 INFO:scrape: 2022-03-08T13:23:35.845 INFO:scrape:Failure: {'smithi117.front.sepia.ceph.com': {'_ansible_no_log': False, '_ansible_verbose_override': True, 'ansible_facts': {}, 'changed': False, 'failed_modules': {'setup': {'failed': True, 'module_stderr': '', 'module_stdout': '', 'msg': 'MODULE FAILURE\nSee stdout/stderr for the exact error', 'rc': -13}}, 'msg': 'The following modules failed to execute: setup\n'}} 2022-03-08T13:23:35.845 INFO:scrape:3 jobs: ['6726615', '6726517', '6726575'] 2022-03-08T13:23:35.846 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'ranks/5', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.846 INFO:scrape:suites union: ['2-workunit/fs/trivial_sync}}', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'races', 'ranks/5', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/iogen}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-03-08T13:23:35.847 INFO:scrape: 2022-03-08T13:23:35.847 INFO:scrape:Failure: SSH connection to smithi038 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' 2022-03-08T13:23:35.847 INFO:scrape:2 jobs: ['6726630', '6726586'] 2022-03-08T13:23:35.848 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.849 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu/{latest', 'fs/verify/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag', 'overrides/{mon-debug', 'overrides}}', 'ranks/3', 'ranks/5', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/fsstress', 'tasks/{0-check-counter', 'validater/lockdep}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/iozone}', 'wsync/{yes}}'] 2022-03-08T13:23:35.849 INFO:scrape: 2022-03-08T13:23:35.849 INFO:scrape:Failure: Test failure: test_client_abort (tasks.cephfs.test_failover.TestFailover) 2022-03-08T13:23:35.850 INFO:scrape:1 jobs: ['6726693'] 2022-03-08T13:23:35.850 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/multifs/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{mon-debug', 'tasks/failover}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.851 INFO:scrape: 2022-03-08T13:23:35.851 INFO:scrape:Failure: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2022-03-08T13:23:35.852 INFO:scrape:2 jobs: ['6726557', '6726815'] 2022-03-08T13:23:35.855 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'mds', 'mon', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{no_client_pidfile', 'tasks/strays}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.855 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'tasks/strays}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.856 INFO:scrape: 2022-03-08T13:23:35.856 INFO:scrape:Failure: "2022-03-08T10:20:48.373812+0000 mds.a (mds.1) 1 : cluster [ERR] error reading sessionmap 'mds1_sessionmap' -2 ((2) No such file or directory)" in cluster log 2022-03-08T13:23:35.857 INFO:scrape:1 jobs: ['6726713'] 2022-03-08T13:23:35.857 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'tasks/journal-repair}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.858 INFO:scrape: 2022-03-08T13:23:35.858 INFO:scrape:Failure: 2022-03-08T13:23:35.859 INFO:scrape:1 jobs: ['6726544'] 2022-03-08T13:23:35.859 INFO:scrape:suites: ['2-workunit/fs/trivial_sync}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'prefetch_dirfrags/yes', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.859 INFO:scrape: 2022-03-08T13:23:35.860 INFO:scrape:Failure: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2022-03-08T13:23:35.860 INFO:scrape:1 jobs: ['6726645'] 2022-03-08T13:23:35.861 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'tasks/volumes/{overrides', 'test/misc}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.861 INFO:scrape: 2022-03-08T13:23:35.862 INFO:scrape:Failure: "2022-03-08T08:20:11.492277+0000 mon.a (mon.0) 1336 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log 2022-03-08T13:23:35.862 INFO:scrape:3 jobs: ['6726543', '6726743', '6726797'] 2022-03-08T13:23:35.863 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{no_client_pidfile', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.863 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/metrics}', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.864 INFO:scrape: 2022-03-08T13:23:35.864 INFO:scrape:Failure: Command failed on smithi066 with status 110: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e48575e-9ecf-11ec-8c35-001a4aab830c -e sha1=372729756c2ba42025d263fce78fb2410486f54f -- bash -c 'ceph orch ps'" 2022-03-08T13:23:35.865 INFO:scrape:1 jobs: ['6726754'] 2022-03-08T13:23:35.865 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.866 INFO:scrape: 2022-03-08T13:23:35.866 INFO:scrape:Failure: Command failed on smithi081 with status 1: "sudo nsenter --net=/var/run/netns/ceph-ns-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id vol_data_isolated --client_mountpoint=/volumes/_nogroup/vol_isolated mnt.0" 2022-03-08T13:23:35.867 INFO:scrape:1 jobs: ['6726592'] 2022-03-08T13:23:35.867 INFO:scrape:suites: ['1-client', '2-upgrade', '3-verify}', 'clusters/1-mds-2-client-micro', 'conf/{client', 'fs/upgrade/volumes/import-legacy/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'tasks/{0-nautilus', 'ubuntu_18.04}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-03-08T13:23:35.867 INFO:scrape: 2022-03-08T13:23:35.881 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/ failed: 12 dead: 7 running: 0 waiting: 0 queued: 0 passed: 309 Fail ================================================================= [6726543] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:21:26 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726543/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726543/ "2022-03-08T08:20:11.492277+0000 mon.a (mon.0) 1336 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6726544] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726544/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726544/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b97d2dc65f34b77a44cb423980c71c1 [6726557] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} ----------------------------------------------------------------- time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726557/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726557/ sentry: https://sentry.ceph.com/organizations/ceph/?query=93f8870f4b0b427a8bcab31d2cb1796a Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) [6726592] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} ----------------------------------------------------------------- time: 00:09:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726592/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726592/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1d011e9ce00842fd902ba11f9d4420e1 Command failed on smithi081 with status 1: "sudo nsenter --net=/var/run/netns/ceph-ns-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id vol_data_isolated --client_mountpoint=/volumes/_nogroup/vol_isolated mnt.0" [6726643] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} ----------------------------------------------------------------- time: 00:42:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726643/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726643/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a0c7349fb82a4affa5bd3d5d5be36e7e Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi070 with status 23: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=56eeb74f63f2a50b371e0c8c7cafef6c7dc248bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/misc/multiple_rsync.sh' [6726645] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726645/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726645/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c27f543832c94a6f80ebf8d8ab41168f Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) [6726693] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:25:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726693/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726693/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7dfa616aff93497fb3cbe53d0db6a71f Test failure: test_client_abort (tasks.cephfs.test_failover.TestFailover) [6726713] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} ----------------------------------------------------------------- time: 00:19:43 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726713/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726713/ "2022-03-08T10:20:48.373812+0000 mds.a (mds.1) 1 : cluster [ERR] error reading sessionmap 'mds1_sessionmap' -2 ((2) No such file or directory)" in cluster log [6726743] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} ----------------------------------------------------------------- time: 00:24:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726743/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726743/ "2022-03-08T10:58:20.839410+0000 mon.a (mon.0) 1535 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log [6726754] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:51:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726754/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726754/ sentry: https://sentry.ceph.com/organizations/ceph/?query=23a3ae67fb6241479030c2fe05013619 Command failed on smithi066 with status 110: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e48575e-9ecf-11ec-8c35-001a4aab830c -e sha1=372729756c2ba42025d263fce78fb2410486f54f -- bash -c 'ceph orch ps'" [6726797] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:20:41 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726797/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726797/ "2022-03-08T11:34:34.595015+0000 mon.a (mon.0) 1062 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY)" in cluster log [6726815] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} ----------------------------------------------------------------- time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726815/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726815/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee819f614cc14bd3b3193b0cebc77e51 Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) Dead ================================================================= [6726517] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} ----------------------------------------------------------------- time: 00:04:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726517/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726517/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d947e7f4920844b4b4df8f9d63505aac {'smithi107.front.sepia.ceph.com': {'_ansible_no_log': False, '_ansible_verbose_override': True, 'ansible_facts': {}, 'changed': False, 'failed_modules': {'setup': {'failed': True, 'module_stderr': '', 'module_stdout': '', 'msg': 'MODULE FAILURE\nSee stdout/stderr for the exact error', 'rc': -13}}, 'msg': 'The following modules failed to execute: setup\n'}} [6726575] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} ----------------------------------------------------------------- time: 00:04:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726575/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726575/ sentry: https://sentry.ceph.com/organizations/ceph/?query=95d78d472f774771a4d8197ec5e5f029 {'smithi081.front.sepia.ceph.com': {'_ansible_no_log': False, '_ansible_verbose_override': True, 'ansible_facts': {}, 'changed': False, 'failed_modules': {'setup': {'failed': True, 'module_stderr': '', 'module_stdout': '', 'msg': 'MODULE FAILURE\nSee stdout/stderr for the exact error', 'rc': -13}}, 'msg': 'The following modules failed to execute: setup\n'}} [6726586] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:32:17 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726586/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726586/ sentry: https://sentry.ceph.com/organizations/ceph/?query=84dba8744768416a86fbaef098d6aac6 SSH connection to smithi023 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' [6726615] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:04:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726615/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726615/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e736a55853f548ac96b350f423b7d402 {'smithi117.front.sepia.ceph.com': {'_ansible_no_log': False, '_ansible_verbose_override': True, 'ansible_facts': {}, 'changed': False, 'failed_modules': {'setup': {'failed': True, 'module_stderr': '', 'module_stdout': '', 'msg': 'MODULE FAILURE\nSee stdout/stderr for the exact error', 'rc': -13}}, 'msg': 'The following modules failed to execute: setup\n'}} [6726630] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} ----------------------------------------------------------------- time: 00:28:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726630/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726630/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03af4d2844a7499d991ebafea067cd2e SSH connection to smithi038 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' [6726735] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726735/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726735/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6726786] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} ----------------------------------------------------------------- time: 00:28:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726786/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726786/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5c44cedd6bc4bba8b9da305ac96eae7 SSH connection to smithi002 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' Pass ================================================================= [6726518] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:16:07 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726518/ [6726519] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:19 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726519/ [6726520] fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726520/ [6726521] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} time: 00:43:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726521/ [6726522] fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full} time: 00:14:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726522/ [6726523] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726523/ [6726524] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} time: 01:57:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726524/ [6726525] fs/mirror-ha/{begin cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} time: 02:37:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726525/ [6726526] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:33:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726526/ [6726527] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:19:18 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726527/ [6726528] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726528/ [6726529] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:12:32 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726529/ [6726530] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:34:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726530/ [6726531] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:17:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726531/ [6726532] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{centos_8} tasks/fstop} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726532/ [6726533] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:26:42 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726533/ [6726534] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 02:07:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726534/ [6726535] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:49:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726535/ [6726536] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:18:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726536/ [6726537] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:19:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726537/ [6726538] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:56:35 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726538/ [6726539] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:35:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726539/ [6726540] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:23:45 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726540/ [6726541] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:40:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726541/ [6726542] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726542/ [6726545] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:20:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726545/ [6726546] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726546/ [6726547] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 01:15:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726547/ [6726548] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726548/ [6726549] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:23:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726549/ [6726550] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:49:54 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726550/ [6726551] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 00:23:26 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726551/ [6726552] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:35:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726552/ [6726553] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726553/ [6726554] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:37:14 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726554/ [6726555] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726555/ [6726556] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:19:48 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726556/ [6726558] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:16:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726558/ [6726559] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:18:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726559/ [6726560] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:52:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726560/ [6726561] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:56:04 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726561/ [6726562] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:22:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726562/ [6726563] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:25:24 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726563/ [6726564] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:16:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726564/ [6726565] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726565/ [6726566] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:16:25 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726566/ [6726567] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:25:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726567/ [6726568] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:23:48 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726568/ [6726569] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:41:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726569/ [6726570] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:02 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726570/ [6726571] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726571/ [6726572] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726572/ [6726573] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:15:42 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726573/ [6726574] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:23:30 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726574/ [6726576] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:19:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726576/ [6726577] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:17:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726577/ [6726578] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726578/ [6726579] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:14:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726579/ [6726580] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726580/ [6726581] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:16:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726581/ [6726582] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726582/ [6726583] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:32:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726583/ [6726584] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:14 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726584/ [6726585] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:52:43 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726585/ [6726587] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:14:59 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726587/ [6726588] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:18:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726588/ [6726589] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726589/ [6726590] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:45:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726590/ [6726591] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:38:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726591/ [6726593] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:22:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726593/ [6726594] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 01:46:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726594/ [6726595] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:15:35 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726595/ [6726596] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726596/ [6726597] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:45:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726597/ [6726598] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:29:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726598/ [6726599] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:22:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726599/ [6726600] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726600/ [6726601] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:12:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726601/ [6726602] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726602/ [6726603] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:49:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726603/ [6726604] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 00:39:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726604/ [6726605] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:21:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726605/ [6726606] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:20:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726606/ [6726607] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726607/ [6726608] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:22:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726608/ [6726609] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726609/ [6726610] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:21:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726610/ [6726611] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb} time: 00:09:30 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726611/ [6726612] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:18:18 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726612/ [6726613] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726613/ [6726614] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:18:54 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726614/ [6726616] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:30:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726616/ [6726617] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726617/ [6726618] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:23:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726618/ [6726619] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:44:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726619/ [6726620] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:45:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726620/ [6726621] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:18:14 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726621/ [6726622] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:15:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726622/ [6726623] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:05:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726623/ [6726624] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:22:54 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726624/ [6726625] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:37:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726625/ [6726626] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726626/ [6726627] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:28:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726627/ [6726628] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:17:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726628/ [6726629] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:15:25 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726629/ [6726631] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:18:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726631/ [6726632] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:25:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726632/ [6726633] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:14:19 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726633/ [6726634] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:15:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726634/ [6726635] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:19:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726635/ [6726636] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:30:49 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726636/ [6726637] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:18:22 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726637/ [6726638] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:23:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726638/ [6726639] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 00:58:04 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726639/ [6726640] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} time: 00:56:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726640/ [6726641] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726641/ [6726642] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 00:50:22 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726642/ [6726644] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726644/ [6726646] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:53:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726646/ [6726647] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726647/ [6726648] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726648/ [6726649] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726649/ [6726650] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:26:17 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726650/ [6726651] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:19:56 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726651/ [6726652] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726652/ [6726653] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726653/ [6726654] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726654/ [6726655] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726655/ [6726656] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 00:44:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726656/ [6726657] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:18:45 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726657/ [6726658] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 01:00:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726658/ [6726659] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:00:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726659/ [6726660] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726660/ [6726661] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:44:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726661/ [6726662] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:17:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726662/ [6726663] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:15:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726663/ [6726664] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726664/ [6726665] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:31:31 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726665/ [6726666] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:15:59 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726666/ [6726667] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:21:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726667/ [6726668] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}} time: 00:21:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726668/ [6726669] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:16:12 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726669/ [6726670] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:23:19 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726670/ [6726671] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} time: 00:15:37 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726671/ [6726672] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:22:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726672/ [6726673] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:25:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726673/ [6726674] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:25:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726674/ [6726675] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:39:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726675/ [6726676] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:59:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726676/ [6726677] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726677/ [6726678] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:56:48 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726678/ [6726679] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:57:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726679/ [6726680] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:27:02 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726680/ [6726681] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:18:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726681/ [6726682] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:14:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726682/ [6726683] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:14:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726683/ [6726684] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:15:43 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726684/ [6726685] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:13:30 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726685/ [6726686] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:43:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726686/ [6726687] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:25:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726687/ [6726688] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:14:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726688/ [6726689] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726689/ [6726690] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726690/ [6726691] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:24:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726691/ [6726692] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:19:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726692/ [6726694] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726694/ [6726695] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726695/ [6726696] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/lockdep} time: 00:19:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726696/ [6726697] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726697/ [6726698] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 00:45:57 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726698/ [6726699] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:58:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726699/ [6726700] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:43:50 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726700/ [6726701] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726701/ [6726702] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:16:09 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726702/ [6726703] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:33:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726703/ [6726704] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:10:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726704/ [6726705] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726705/ [6726706] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726706/ [6726707] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:19:30 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726707/ [6726708] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:18:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726708/ [6726709] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:38:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726709/ [6726710] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:17:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726710/ [6726711] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:50:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726711/ [6726712] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:37 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726712/ [6726714] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 00:49:35 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726714/ [6726715] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726715/ [6726716] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:56:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726716/ [6726717] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:19:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726717/ [6726718] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:16:43 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726718/ [6726719] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726719/ [6726720] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:14:36 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726720/ [6726721] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:20:54 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726721/ [6726722] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726722/ [6726723] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726723/ [6726724] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:15:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726724/ [6726725] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:25:22 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726725/ [6726726] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:40:15 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726726/ [6726727] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:22:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726727/ [6726728] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:44:21 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726728/ [6726729] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:20:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726729/ [6726730] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:18:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726730/ [6726731] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726731/ [6726732] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 01:02:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726732/ [6726733] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:16:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726733/ [6726734] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:12:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726734/ [6726736] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:21:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726736/ [6726737] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:29:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726737/ [6726738] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726738/ [6726739] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:19:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726739/ [6726740] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:23:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726740/ [6726741] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:16:04 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726741/ [6726742] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726742/ [6726744] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:34:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726744/ [6726745] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726745/ [6726746] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:25:57 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726746/ [6726747] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:14:41 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726747/ [6726748] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/valgrind} time: 00:55:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726748/ [6726749] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726749/ [6726750] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:20:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726750/ [6726751] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 00:39:22 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726751/ [6726752] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:33:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726752/ [6726753] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726753/ [6726755] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:37:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726755/ [6726756] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726756/ [6726757] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726757/ [6726758] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:30:42 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726758/ [6726759] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726759/ [6726760] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:16:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726760/ [6726761] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 01:19:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726761/ [6726762] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:15:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726762/ [6726763] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}} time: 00:30:43 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726763/ [6726764] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:20:28 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726764/ [6726765] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726765/ [6726766] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:36:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726766/ [6726767] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:18:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726767/ [6726768] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:17:46 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726768/ [6726769] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726769/ [6726770] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726770/ [6726771] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726771/ [6726772] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:20:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726772/ [6726773] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:18:06 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726773/ [6726774] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:10:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726774/ [6726775] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726775/ [6726776] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:23:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726776/ [6726777] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:29:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726777/ [6726778] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:16:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726778/ [6726779] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726779/ [6726780] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:41:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726780/ [6726781] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:18:58 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726781/ [6726782] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:58:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726782/ [6726783] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:25:27 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726783/ [6726784] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:14:41 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726784/ [6726785] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726785/ [6726787] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:21:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726787/ [6726788] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:16:45 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726788/ [6726789] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726789/ [6726790] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:14:05 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726790/ [6726791] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:02 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726791/ [6726792] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:15:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726792/ [6726793] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:49:47 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726793/ [6726794] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:17:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726794/ [6726795] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726795/ [6726796] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:18:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726796/ [6726798] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:54:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726798/ [6726799] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:11:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726799/ [6726800] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:49:05 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726800/ [6726801] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} time: 00:32:37 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726801/ [6726802] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} time: 00:16:39 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726802/ [6726803] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:05:17 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726803/ [6726804] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} time: 00:41:54 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726804/ [6726805] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:44:53 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726805/ [6726806] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:22:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726806/ [6726807] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:18:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726807/ [6726808] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:37:36 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726808/ [6726809] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:10:23 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726809/ [6726810] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:20:01 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726810/ [6726811] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:22:55 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726811/ [6726812] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726812/ [6726813] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726813/ [6726814] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:23:37 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726814/ [6726816] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:20 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726816/ [6726817] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} time: 01:16:13 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726817/ [6726818] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:21:50 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726818/ [6726819] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:40:41 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726819/ [6726820] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:37:34 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726820/ [6726821] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:19:41 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726821/ [6726822] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:16:08 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726822/ [6726823] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:32:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726823/ [6726824] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:20:49 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726824/ [6726825] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726825/ [6726826] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726826/ [6726827] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:14:17 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726827/ [6726828] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:43:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726828/ [6726829] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:27:03 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726829/ [6726830] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:15:51 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726830/ [6726831] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:42 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726831/ [6726832] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726832/ [6726833] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:14:16 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726833/ [6726834] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:07:19 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726834/ [6726835] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726835/ [6726836] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} time: 00:24:07 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726836/ [6726837] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:38:14 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726837/ [6726838] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{no}} time: 00:26:33 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726838/ [6726839] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:18:40 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726839/ [6726840] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:16:29 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726840/ [6726841] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:12:24 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726841/ [6726842] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:20:44 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726842/ [6726843] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726843/ [6726844] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2022-03-08_04:17:02-fs-pacific-distro-default-smithi/6726844/