2022-04-04T17:11:41.402 INFO:root:teuthology version: 1.1.0-d877b023 2022-04-04T17:11:41.404 INFO:teuthology.results:subset: '111/120000' 2022-04-04T17:11:41.404 INFO:teuthology.results:seed: '7600' 2022-04-04T19:30:53.991 INFO:root:teuthology version: 1.1.0-d877b023 2022-04-04T19:30:54.021 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-05T00:31:18.605 INFO:teuthology.results:Tests finished! gathering results... 2022-04-05T00:31:35.917 INFO:scrape:Found 181 jobs 2022-04-05T00:31:52.939 INFO:scrape:Found 12 distinct failure reasons 2022-04-05T00:31:52.940 INFO:scrape:Failure: Command failed on smithi036 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" 2022-04-05T00:31:52.941 INFO:scrape:24 jobs: ['6776308', '6776341', '6776389', '6776331', '6776478', '6776458', '6776407', '6776336', '6776446', '6776415', '6776320', '6776423', '6776354', '6776379', '6776403', '6776327', '6776304', '6776363', '6776474', '6776465', '6776429', '6776394', '6776434', '6776371'] 2022-04-05T00:31:52.942 INFO:scrape:suites intersection: [] 2022-04-05T00:31:52.942 INFO:scrape:suites union: ['1-client', '1-mount', '1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-upgrade', '2-workload/dbench', '3-client-upgrade', '3-compat_client/no}}', '3-compat_client/pacific}}', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '4-compat_client', '4-verify}', '4-verify}}', '5-client-sanity}}', 'centos_8.stream_container_tools', 'centos_latest', 'clusters/1-mds-2-client-micro', 'conf/{client', 'distro/single-container-host}', 'fs/cephadm/multivolume/{0-start', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{multimds/no', 'overrides/{multimds/yes', 'overrides/{pg-warn', 'pg-warn', 'roles', 'syntax', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'tasks/{0-octopus', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.943 INFO:scrape: 2022-04-05T00:31:52.943 INFO:scrape:Failure: "2022-04-04T18:29:44.202420+0000 mon.a (mon.0) 1510 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log 2022-04-05T00:31:52.944 INFO:scrape:3 jobs: ['6776383', '6776475', '6776435'] 2022-04-05T00:31:52.944 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{no_client_pidfile', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.945 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'fs/volumes/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/admin}', 'tasks/snap-schedule}', 'tasks/volumes/{overrides', 'test/snapshot}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.945 INFO:scrape: 2022-04-05T00:31:52.946 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2022-04-05T00:31:52.947 INFO:scrape:1 jobs: ['6776421'] 2022-04-05T00:31:52.947 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/multifs/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.948 INFO:scrape: 2022-04-05T00:31:52.948 INFO:scrape:Failure: Command failed on smithi025 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid 55a2649e-b444-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.25 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' 2022-04-05T00:31:52.949 INFO:scrape:5 jobs: ['6776376', '6776334', '6776448', '6776412', '6776457'] 2022-04-05T00:31:52.949 INFO:scrape:suites intersection: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.950 INFO:scrape:suites union: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'ranks/1', 'ranks/5', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/direct_io}', 'workunit/fs/test_o_trunc}', 'workunit/suites/dbench}', 'workunit/suites/fsstress}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}'] 2022-04-05T00:31:52.950 INFO:scrape: 2022-04-05T00:31:52.951 INFO:scrape:Failure: Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring) 2022-04-05T00:31:52.951 INFO:scrape:1 jobs: ['6776355'] 2022-04-05T00:31:52.952 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}'] 2022-04-05T00:31:52.952 INFO:scrape: 2022-04-05T00:31:52.952 INFO:scrape:Dead: b'2022-04-05T00:23:03.245 DEBUG:teuthology.exit:Finished running handlers' 2022-04-05T00:31:52.953 INFO:scrape:ceph version 17.0.0-11448-g0e054242 (0e054242b38b7ba9dcd21d35b3a42e5b2f85d681) quincy (dev) 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fd618276ce0] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b8) [0x559de57dd19e] 5: (KernelDevice::_aio_thread()+0x1275) [0x559de642fae5] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x559de643d041] 7: /lib64/libpthread.so.0(+0x81cf) [0x7fd61826c1cf] 8: clone() 2022-04-05T00:31:52.953 INFO:scrape:1 jobs: ['6776357'] 2022-04-05T00:31:52.954 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/multifs/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.954 INFO:scrape: 2022-04-05T00:31:52.955 INFO:scrape:Failure: Command failed (workunit test suites/iozone.sh) on smithi040 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' 2022-04-05T00:31:52.955 INFO:scrape:3 jobs: ['6776333', '6776447', '6776378'] 2022-04-05T00:31:52.956 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distro/{centos_8}', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.956 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/iozone}}', '2-workunit/suites/iozone}}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.957 INFO:scrape: 2022-04-05T00:31:52.957 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1649097683.340:179): avc: denied { node_bind } for pid=1407 comm="ping" saddr=172.21.15.6 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2022-04-05T00:31:52.958 INFO:scrape:2 jobs: ['6776413', '6776392'] 2022-04-05T00:31:52.958 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'mds', 'mon', 'mount', 'osd}', 'ranks/3', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.959 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/suites/ffsb}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'fs/verify/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'overrides/{mon-debug', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/no', 'races', 'ranks/3', 'session_timeout', 'tasks/fsstress', 'tasks/{1-thrash/mds', 'thrashosds-health', 'validater/lockdep}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.960 INFO:scrape: 2022-04-05T00:31:52.960 INFO:scrape:Failure: Test failure: test_perf_stats_stale_metrics (tasks.cephfs.test_mds_metrics.TestMDSMetrics) 2022-04-05T00:31:52.960 INFO:scrape:1 jobs: ['6776404'] 2022-04-05T00:31:52.961 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{no_client_pidfile', 'tasks/metrics}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.961 INFO:scrape: 2022-04-05T00:31:52.962 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) 2022-04-05T00:31:52.962 INFO:scrape:2 jobs: ['6776467', '6776315'] 2022-04-05T00:31:52.963 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{no_client_pidfile', 'tasks/acls}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.963 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/acls}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.964 INFO:scrape: 2022-04-05T00:31:52.964 INFO:scrape:Failure: SSH connection to smithi081 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' 2022-04-05T00:31:52.965 INFO:scrape:1 jobs: ['6776460'] 2022-04-05T00:31:52.965 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{mon-debug', 'overrides}}', 'ranks/1', 'session_timeout', 'tasks/fsstress', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.966 INFO:scrape: 2022-04-05T00:31:52.966 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi025 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh' 2022-04-05T00:31:52.967 INFO:scrape:1 jobs: ['6776470'] 2022-04-05T00:31:52.967 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/snaps/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{whitelist_health', 'rhel_8}', 'tasks/workunit/snaps}', 'whitelist_wrongly_marked_down}'] 2022-04-05T00:31:52.968 INFO:scrape: 2022-04-05T00:31:52.977 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/ failed: 43 dead: 2 running: 0 waiting: 0 queued: 0 passed: 136 Fail ================================================================= [6776304] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776304/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776304/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8be0ad3dab84bd7b4f273596dfd6109 Command failed on smithi173 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776308] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:08:41 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776308/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d848709313314ea793e6c57d6871f567 Command failed on smithi036 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776315] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776315/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776315/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8b804ad286744c5bbecccb57d3bf97f Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) [6776320] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:01 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776320/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776320/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b15313e5a674d28a648441ecfac72bd Command failed on smithi146 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776327] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:18 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776327/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776327/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6a474568e88343c1bb17af6e16d69006 Command failed on smithi184 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776331] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:14:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776331/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776331/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16afb078a28b4406a98332904715a73e Command failed on smithi100 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776333] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776333/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776333/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5684c9ac16df42f08279ca53354425c3 Command failed (workunit test suites/iozone.sh) on smithi040 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [6776334] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} ----------------------------------------------------------------- time: 00:10:52 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776334/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776334/ sentry: https://sentry.ceph.com/organizations/ceph/?query=742e97a6b59049c5ac68022442aa6598 Command failed on smithi005 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid 837cba9e-b440-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [6776336] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:08:42 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776336/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776336/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40c9bfead23941f0afc5d0ca05f49d99 Command failed on smithi190 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776341] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} ----------------------------------------------------------------- time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776341/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776341/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8cdde37220047e9b2110c76871042fb Command failed on smithi045 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776354] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776354/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776354/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3585daef21f410a88009496b2a1cf1f Command failed on smithi134 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776355] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} ----------------------------------------------------------------- time: 00:15:02 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776355/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776355/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc6d313c65ec4db7b1eee01b93876bfb Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring) [6776363] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776363/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776363/ sentry: https://sentry.ceph.com/organizations/ceph/?query=266517c02b4e41e4b5b0607e1a45d4d1 Command failed on smithi043 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776371] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:12:27 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776371/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776371/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3727dedb852b43478e8391c6d51c3587 Command failed on smithi062 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776376] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} ----------------------------------------------------------------- time: 00:11:01 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776376/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776376/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d69c0807cbf44089868bb02bc7479d8c Command failed on smithi025 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid 55a2649e-b444-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.25 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [6776378] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:10:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776378/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776378/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3de70b7bd47d4d6ca1e4ad3b5249ce56 Command failed (workunit test suites/iozone.sh) on smithi039 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [6776379] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776379/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776379/ sentry: https://sentry.ceph.com/organizations/ceph/?query=726825c93866464cb2c42c566909780e Command failed on smithi142 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776383] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:20:22 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776383/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776383/ "2022-04-04T18:29:44.202420+0000 mon.a (mon.0) 1510 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [6776389] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:08:09 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776389/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776389/ sentry: https://sentry.ceph.com/organizations/ceph/?query=504aba3f06fe4024b44e7366007f679f Command failed on smithi178 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776392] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776392/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776392/ SELinux denials found on ubuntu@smithi131.front.sepia.ceph.com: ['type=AVC msg=audit(1649096862.191:186): avc: denied { node_bind } for pid=1468 comm="ping" saddr=172.21.15.131 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6776394] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:14:13 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776394/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776394/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ab62725382bd4cad9bba70ad97d7d24b Command failed on smithi195 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776403] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776403/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776403/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9c313019c6264a789c3a53d5d69c34a6 Command failed on smithi146 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776404] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} ----------------------------------------------------------------- time: 00:19:59 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776404/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776404/ sentry: https://sentry.ceph.com/organizations/ceph/?query=172208dd07144e4cb0b1b91220f79e1f Test failure: test_perf_stats_stale_metrics (tasks.cephfs.test_mds_metrics.TestMDSMetrics) [6776407] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:15 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776407/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776407/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2314e2cd97294b398a748f572786905e Command failed on smithi173 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776412] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} ----------------------------------------------------------------- time: 00:10:34 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776412/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776412/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f990921b17d43bf9b32870e4d5786dd Command failed on smithi117 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid 1ff6660c-b448-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.117 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [6776413] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:03:28 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776413/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776413/ SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1649097683.340:179): avc: denied { node_bind } for pid=1407 comm="ping" saddr=172.21.15.6 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] [6776415] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776415/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776415/ sentry: https://sentry.ceph.com/organizations/ceph/?query=92c772e22734453ca0c3fb44af460c67 Command failed on smithi170 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776421] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:28:08 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776421/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776421/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e508ae48c4942a1a9d287c3c4845cd1 Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git- ceph.sh' [6776423] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776423/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776423/ sentry: https://sentry.ceph.com/organizations/ceph/?query=092448b50e794b4baa9fc8f0bbc1ab3b Command failed on smithi146 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776429] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776429/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776429/ sentry: https://sentry.ceph.com/organizations/ceph/?query=22a34e38aaa44cceb14028cf27f18a16 Command failed on smithi173 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776434] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:10 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776434/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776434/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01f588ed739f4f838528ef90a0016c8f Command failed on smithi045 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776435] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:44:46 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776435/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776435/ "2022-04-04T19:03:54.180067+0000 mon.a (mon.0) 451 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6776446] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 00:08:12 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776446/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776446/ sentry: https://sentry.ceph.com/organizations/ceph/?query=68267f1674004ff1bf18c2f641b5a139 Command failed on smithi169 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776447] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:10:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776447/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776447/ sentry: https://sentry.ceph.com/organizations/ceph/?query=75f7a7fe06e149efa1e34b5beead9a6e Command failed (workunit test suites/iozone.sh) on smithi057 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [6776448] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} ----------------------------------------------------------------- time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776448/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776448/ sentry: https://sentry.ceph.com/organizations/ceph/?query=db7c13249639467e87793b3ad4b02253 Command failed on smithi017 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid ff963e1a-b44b-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.17 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [6776457] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} ----------------------------------------------------------------- time: 00:10:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776457/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776457/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6043d99f199f4d8aa64dc4facbf39323 Command failed on smithi105 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 -v bootstrap --fsid fa80f432-b44c-11ec-8c36-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial- daemons --skip-monitoring-stack --mon-ip 172.21.15.105 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring' [6776458] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:14:11 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776458/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776458/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7bf390d46c14740ad77b0f8077ecc7e Command failed on smithi045 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776465] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776465/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776465/ sentry: https://sentry.ceph.com/organizations/ceph/?query=edd414e108be4626a383ae3817c67bd6 Command failed on smithi050 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.0" [6776467] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:15:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776467/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776467/ sentry: https://sentry.ceph.com/organizations/ceph/?query=545f57516c5945f2a0187004d7e6dcf5 Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) [6776470] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:18:03 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776470/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776470/ sentry: https://sentry.ceph.com/organizations/ceph/?query=633df6a794a94222a870b5cab0834dae Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi025 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0e054242b38b7ba9dcd21d35b3a42e5b2f85d681 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest- multiple-capsnaps.sh' [6776474] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:12:05 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776474/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776474/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aa5fdf87aebe43d399653d95d2ea25df Command failed on smithi146 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" [6776475] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} ----------------------------------------------------------------- time: 00:48:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776475/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776475/ "2022-04-04T19:44:38.776279+0000 mon.a (mon.0) 1835 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [6776478] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776478/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776478/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e5a44c1511f4207ad917bc7741eb119 Command failed on smithi122 with status 2: "cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\\0' >/home/ubuntu/cephtest/workunits.list.client.1" Dead ================================================================= [6776357] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776357/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776357/ hit max job timeout [6776460] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 01:07:01 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776460/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776460/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c0bef5c11194674a6a570f826db4e6a SSH connection to smithi081 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' Pass ================================================================= [6776305] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:10:27 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776305/ [6776306] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:12:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776306/ [6776307] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776307/ [6776309] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776309/ [6776310] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 02:08:16 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776310/ [6776311] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:00:21 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776311/ [6776312] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:13:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776312/ [6776313] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:09:18 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776313/ [6776314] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} time: 00:14:45 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776314/ [6776316] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776316/ [6776317] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:45:23 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776317/ [6776318] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:58:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776318/ [6776319] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:42:04 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776319/ [6776321] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776321/ [6776322] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:52:11 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776322/ [6776323] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776323/ [6776324] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:10:18 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776324/ [6776325] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} time: 00:45:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776325/ [6776326] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 02:47:27 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776326/ [6776328] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:08:14 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776328/ [6776329] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:20:24 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776329/ [6776330] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 00:41:22 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776330/ [6776332] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:12:40 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776332/ [6776335] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:07:29 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776335/ [6776337] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 01:05:26 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776337/ [6776338] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:10:05 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776338/ [6776339] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776339/ [6776340] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137} time: 00:08:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776340/ [6776342] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776342/ [6776343] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:07:48 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776343/ [6776344] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 01:54:04 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776344/ [6776345] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:19:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776345/ [6776346] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:40:09 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776346/ [6776347] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776347/ [6776348] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:27:40 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776348/ [6776349] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:09:35 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776349/ [6776350] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:31:44 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776350/ [6776351] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:54:15 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776351/ [6776352] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{ubuntu_latest} tasks/fstop} time: 00:07:20 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776352/ [6776353] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:20:10 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776353/ [6776356] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776356/ [6776358] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776358/ [6776359] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:56:49 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776359/ [6776360] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:50:29 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776360/ [6776361] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:07:28 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776361/ [6776362] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:19:52 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776362/ [6776364] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:28:05 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776364/ [6776365] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:17:24 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776365/ [6776366] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}} time: 00:24:20 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776366/ [6776367] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:19:13 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776367/ [6776368] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:07:25 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776368/ [6776369] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776369/ [6776370] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:22:48 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776370/ [6776372] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:01 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776372/ [6776373] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776373/ [6776374] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:40:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776374/ [6776375] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:14:11 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776375/ [6776377] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776377/ [6776380] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:08:34 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776380/ [6776381] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:46:58 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776381/ [6776382] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:47:47 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776382/ [6776384] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776384/ [6776385] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:48:56 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776385/ [6776386] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:45:55 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776386/ [6776387] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776387/ [6776388] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776388/ [6776390] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:35:42 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776390/ [6776391] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776391/ [6776393] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:09:15 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776393/ [6776395] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:12:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776395/ [6776396] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} time: 00:19:38 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776396/ [6776397] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb} time: 00:07:52 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776397/ [6776398] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:12:24 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776398/ [6776399] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 01:00:07 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776399/ [6776400] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:12:11 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776400/ [6776401] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} time: 00:21:29 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776401/ [6776402] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:36:52 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776402/ [6776405] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776405/ [6776406] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776406/ [6776408] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776408/ [6776409] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:21:40 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776409/ [6776410] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:36:44 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776410/ [6776411] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:56:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776411/ [6776414] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:13:23 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776414/ [6776416] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:09:56 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776416/ [6776417] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:09:46 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776417/ [6776418] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:47:25 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776418/ [6776419] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776419/ [6776420] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:44:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776420/ [6776422] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:08:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776422/ [6776424] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:36:00 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776424/ [6776425] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776425/ [6776426] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs} time: 00:10:19 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776426/ [6776427] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} time: 00:51:21 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776427/ [6776428] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:14:28 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776428/ [6776430] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:21:57 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776430/ [6776431] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776431/ [6776432] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776432/ [6776433] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} time: 00:22:53 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776433/ [6776436] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:22 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776436/ [6776437] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/whitelist_health} time: 00:13:51 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776437/ [6776438] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776438/ [6776439] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:22 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776439/ [6776440] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:49:32 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776440/ [6776441] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:09:38 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776441/ [6776442] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:13:49 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776442/ [6776443] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:39:05 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776443/ [6776444] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776444/ [6776445] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:23:30 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776445/ [6776449] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:25:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776449/ [6776450] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:50:08 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776450/ [6776451] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:29 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776451/ [6776452] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776452/ [6776453] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:07:13 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776453/ [6776454] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:14:49 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776454/ [6776455] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:10:09 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776455/ [6776456] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:10:52 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776456/ [6776459] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776459/ [6776461] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776461/ [6776462] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:09:57 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776462/ [6776463] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:55:39 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776463/ [6776464] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776464/ [6776466] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:36:14 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776466/ [6776468] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:10 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776468/ [6776469] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:20:06 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776469/ [6776471] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:14 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776471/ [6776472] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 01:13:57 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776472/ [6776473] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:13:35 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776473/ [6776476] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:10:06 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776476/ [6776477] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:45:06 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776477/ [6776479] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}} time: 00:42:39 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776479/ [6776480] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:23:10 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776480/ [6776481] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:09:13 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776481/ [6776482] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:33 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776482/ [6776483] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:26:43 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776483/ [6776484] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:09 info: http://pulpito.ceph.com/yuriw-2022-04-04_17:00:37-fs-wip-yuri-testing-2022-04-04-0731-distro-default-smithi/6776484/