2021-03-25T02:46:59.200 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-25T02:46:59.200 INFO:teuthology.results:subset: '111/120000' 2021-03-25T02:46:59.201 INFO:teuthology.results:seed: '908' 2021-03-25T05:35:22.078 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-25T05:35:22.110 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-25T17:36:18.920 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-03-25T17:36:27.575 INFO:scrape:Found 155 jobs 2021-03-25T17:37:27.124 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:27.125 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:27.352 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:27.352 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:27.579 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:27.579 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:27.812 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:27.813 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:28.043 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:28.044 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:28.270 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:28.270 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:28.495 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:28.495 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:28.725 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:28.725 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:28.959 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:28.959 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:29.191 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:29.192 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:29.426 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:29.426 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:29.655 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:29.655 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:29.890 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:29.890 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:30.133 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:30.133 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:30.369 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:30.370 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:30.606 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:30.607 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:30.834 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:30.834 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:31.068 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:31.069 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:31.300 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:31.300 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:31.522 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:31.522 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:31.743 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:31.743 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:31.965 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:31.966 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:32.191 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:32.192 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:32.422 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:32.422 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:32.656 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:32.656 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:32.896 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:32.897 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:33.118 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:33.118 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:33.339 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:33.339 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:33.554 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:33.555 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:33.774 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:33.774 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:34.008 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:34.009 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:34.229 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:34.229 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:34.454 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:34.454 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:34.678 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:34.678 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:34.906 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:34.906 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:35.123 WARNING:scrape:Not-understood crash indication 2021-03-25T05:06:30.730 INFO:teuthology.orchestra.run.smithi032.stderr:daemon-helper: command crashed with signal 9 2021-03-25T17:37:35.124 WARNING:scrape:Not-understood crash indication 2021-03-25T17:37:35.176 INFO:scrape:Found 36 distinct failure reasons 2021-03-25T17:37:35.177 INFO:scrape:Failure: Test failure: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2021-03-25T17:37:35.177 INFO:scrape:1 jobs: ['5995366'] 2021-03-25T17:37:35.177 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.178 INFO:scrape: 2021-03-25T17:37:35.178 INFO:scrape:Failure: Command failed on smithi031 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' 2021-03-25T17:37:35.178 INFO:scrape:14 jobs: ['5995412', '5995348', '5995374', '5995418', '5995332', '5995337', '5995402', '5995297', '5995353', '5995301', '5995388', '5995382', '5995303', '5995273'] 2021-03-25T17:37:35.178 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.178 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_snaptests}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '2-workunit/fs/trivial_sync}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/fsstress}}', '2-workunit/suites/iozone}}', '2-workunit/suites/pjd}}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/thrash/multifs/{begin', 'fs/thrash/workloads/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/1', 'ranks/3', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}', 'workunit/suites/ffsb}}', 'workunit/suites/fsync-tester}}'] 2021-03-25T17:37:35.179 INFO:scrape: 2021-03-25T17:37:35.179 INFO:scrape:Failure: Test failure: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2021-03-25T17:37:35.179 INFO:scrape:1 jobs: ['5995271'] 2021-03-25T17:37:35.179 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/multimds_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.179 INFO:scrape: 2021-03-25T17:37:35.179 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2021-03-25T17:37:35.179 INFO:scrape:1 jobs: ['5995308'] 2021-03-25T17:37:35.179 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/1', 'scrub/no', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/kernel_untar_build}}'] 2021-03-25T17:37:35.179 INFO:scrape: 2021-03-25T17:37:35.180 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi074 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2021-03-25T17:37:35.180 INFO:scrape:2 jobs: ['5995405', '5995380'] 2021-03-25T17:37:35.180 INFO:scrape:suites intersection: ['clusters/1a3s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/snaps/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{frag_enable', 'tasks/workunit/snaps}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.180 INFO:scrape:suites union: ['clusters/1a3s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/snaps/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/workunit/snaps}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.180 INFO:scrape: 2021-03-25T17:37:35.180 INFO:scrape:Failure: Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi073 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' 2021-03-25T17:37:35.180 INFO:scrape:1 jobs: ['5995358'] 2021-03-25T17:37:35.180 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/3', 'scrub/no', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/norstats}}'] 2021-03-25T17:37:35.180 INFO:scrape: 2021-03-25T17:37:35.181 INFO:scrape:Failure: Test failure: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2021-03-25T17:37:35.181 INFO:scrape:1 jobs: ['5995305'] 2021-03-25T17:37:35.181 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/sessionmap}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.181 INFO:scrape: 2021-03-25T17:37:35.181 INFO:scrape:Failure: Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration) 2021-03-25T17:37:35.181 INFO:scrape:1 jobs: ['5995331'] 2021-03-25T17:37:35.181 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/test_journal_migration}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.181 INFO:scrape: 2021-03-25T17:37:35.181 INFO:scrape:Failure: Command failed (workunit test suites/iozone.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' 2021-03-25T17:37:35.181 INFO:scrape:1 jobs: ['5995251'] 2021-03-25T17:37:35.182 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/5', 'scrub/no', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/iozone}}'] 2021-03-25T17:37:35.182 INFO:scrape: 2021-03-25T17:37:35.182 INFO:scrape:Failure: Command failed (workunit test suites/dbench.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh' 2021-03-25T17:37:35.182 INFO:scrape:2 jobs: ['5995345', '5995287'] 2021-03-25T17:37:35.182 INFO:scrape:suites intersection: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.182 INFO:scrape:suites union: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'kclient-overrides/{distro/testing/{flavor/ubuntu_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.182 INFO:scrape: 2021-03-25T17:37:35.182 INFO:scrape:Failure: 'NoneType' object has no attribute 'mon_manager' 2021-03-25T17:37:35.182 INFO:scrape:4 jobs: ['5995363', '5995387', '5995255', '5995296'] 2021-03-25T17:37:35.183 INFO:scrape:suites intersection: ['1-client', '2-upgrade', 'clusters/1-mds-2-client-micro', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'pg-warn', 'tasks/{0-nautilus', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.183 INFO:scrape:suites union: ['1-client', '2-upgrade', '3-client-upgrade', '3-compat_client/pacific}}', '4-compat_client', '5-client-sanity}}', 'clusters/1-mds-2-client-micro', 'conf/{client', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'mds', 'mon', 'multimds/no', 'multimds/yes', 'osd}', 'overrides/{frag_enable', 'pg-warn', 'tasks/{0-nautilus', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.183 INFO:scrape: 2021-03-25T17:37:35.183 INFO:scrape:Failure: Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation) 2021-03-25T17:37:35.183 INFO:scrape:1 jobs: ['5995398'] 2021-03-25T17:37:35.183 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/fragment}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.183 INFO:scrape: 2021-03-25T17:37:35.183 INFO:scrape:Failure: Test failure: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2021-03-25T17:37:35.183 INFO:scrape:1 jobs: ['5995364'] 2021-03-25T17:37:35.183 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/cap-flush}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.184 INFO:scrape: 2021-03-25T17:37:35.184 INFO:scrape:Failure: Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2021-03-25T17:37:35.184 INFO:scrape:1 jobs: ['5995378'] 2021-03-25T17:37:35.184 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_8}', 'fs/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/multifs-auth}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.184 INFO:scrape: 2021-03-25T17:37:35.184 INFO:scrape:Failure: "2021-03-25T05:14:45.545810+0000 mds.a (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000003b1 (/client.0/tmp/fsstress/ltp-full-20091231/testcases/commands) see mds.a log and `damage ls` output for details" in cluster log 2021-03-25T17:37:35.184 INFO:scrape:3 jobs: ['5995367', '5995413', '5995330'] 2021-03-25T17:37:35.184 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.184 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/3', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/mds_creation_retry}', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}}', 'workunit/suites/fsstress}}'] 2021-03-25T17:37:35.184 INFO:scrape: 2021-03-25T17:37:35.185 INFO:scrape:Failure: Command failed (workunit test suites/fsstress.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' 2021-03-25T17:37:35.185 INFO:scrape:3 jobs: ['5995283', '5995355', '5995414'] 2021-03-25T17:37:35.185 INFO:scrape:suites intersection: ['centos_latest', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'tasks/fsstress', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.185 INFO:scrape:suites union: ['centos_latest', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{frag_enable', 'ranks/1', 'ranks/3', 'ranks/5', 'session_timeout', 'tasks/fsstress', 'validater/lockdep}', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.185 INFO:scrape: 2021-03-25T17:37:35.185 INFO:scrape:Failure: Command failed (workunit test fs/quota/quota.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/quota/quota.sh' 2021-03-25T17:37:35.185 INFO:scrape:1 jobs: ['5995335'] 2021-03-25T17:37:35.185 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/workunit/quota}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.185 INFO:scrape: 2021-03-25T17:37:35.185 INFO:scrape:Crash: Command failed on smithi042 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' 2021-03-25T17:37:35.186 INFO:scrape:ceph version 16.1.0-1201-ga3a37f39 (a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a) pacific (rc) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f8b119b4dc0] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b6) [0x7f8b12c08b59] 5: (operator<<(std::ostream&, ClientMetricType const&)+0xcb) [0x7f8b12e8b02b] 6: (MClientMetrics::print(std::ostream&) const+0x1e9) [0x7f8b12e8b249] 7: (DispatchQueue::pre_dispatch(boost::intrusive_ptr const&)+0x710) [0x7f8b12e3f410] 8: (DispatchQueue::entry()+0xdeb) [0x7f8b12e40e7b] 9: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8b12eef901] 10: /lib64/libpthread.so.0(+0x82de) [0x7f8b119aa2de] 11: clone() 2021-03-25T17:37:35.186 INFO:scrape:5 jobs: ['5995285', '5995391', '5995426', '5995365', '5995342'] 2021-03-25T17:37:35.186 INFO:scrape:suites intersection: ['conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.186 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_suites_fsstress}}', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/fs/snaps}}', '2-workunit/iozone}}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/thrash/multifs/{begin', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/1', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.186 INFO:scrape: 2021-03-25T17:37:35.186 INFO:scrape:Failure: Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) 2021-03-25T17:37:35.186 INFO:scrape:1 jobs: ['5995404'] 2021-03-25T17:37:35.186 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/failover}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.187 INFO:scrape: 2021-03-25T17:37:35.187 INFO:scrape:Failure: Command failed (workunit test fs/test_o_trunc.sh) on smithi073 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh' 2021-03-25T17:37:35.187 INFO:scrape:1 jobs: ['5995399'] 2021-03-25T17:37:35.187 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/5', 'scrub/no', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/test_o_trunc}}'] 2021-03-25T17:37:35.187 INFO:scrape: 2021-03-25T17:37:35.187 INFO:scrape:Failure: Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks), test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2021-03-25T17:37:35.187 INFO:scrape:1 jobs: ['5995302'] 2021-03-25T17:37:35.187 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/scrub}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.187 INFO:scrape: 2021-03-25T17:37:35.187 INFO:scrape:Failure: Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull) 2021-03-25T17:37:35.187 INFO:scrape:2 jobs: ['5995410', '5995253'] 2021-03-25T17:37:35.188 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.188 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.188 INFO:scrape: 2021-03-25T17:37:35.188 INFO:scrape:Failure: Test failure: test_add_data_pool_non_alphamueric_name_as_subdir (tasks.cephfs.test_admin.TestAdminCommands) 2021-03-25T17:37:35.188 INFO:scrape:1 jobs: ['5995344'] 2021-03-25T17:37:35.188 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/admin}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.188 INFO:scrape: 2021-03-25T17:37:35.188 INFO:scrape:Failure: Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 2021-03-25T17:37:35.188 INFO:scrape:2 jobs: ['5995281', '5995425'] 2021-03-25T17:37:35.188 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/openfiletable}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.189 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/openfiletable}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.189 INFO:scrape: 2021-03-25T17:37:35.189 INFO:scrape:Failure: Test failure: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) 2021-03-25T17:37:35.189 INFO:scrape:1 jobs: ['5995361'] 2021-03-25T17:37:35.189 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/backtrace}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.189 INFO:scrape: 2021-03-25T17:37:35.189 INFO:scrape:Failure: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2021-03-25T17:37:35.189 INFO:scrape:1 jobs: ['5995389'] 2021-03-25T17:37:35.189 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/misc}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.189 INFO:scrape: 2021-03-25T17:37:35.189 INFO:scrape:Failure: Test failure: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2021-03-25T17:37:35.190 INFO:scrape:1 jobs: ['5995293'] 2021-03-25T17:37:35.190 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/snapshot}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.190 INFO:scrape: 2021-03-25T17:37:35.190 INFO:scrape:Failure: Test failure: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2021-03-25T17:37:35.190 INFO:scrape:1 jobs: ['5995392'] 2021-03-25T17:37:35.190 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/forward-scrub}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.190 INFO:scrape: 2021-03-25T17:37:35.190 INFO:scrape:Failure: reached maximum tries (150) after waiting for 900 seconds 2021-03-25T17:37:35.190 INFO:scrape:1 jobs: ['5995424'] 2021-03-25T17:37:35.191 INFO:scrape:suites: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'mds', 'mon', 'mount/fuse', 'msgr-failures/none', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'ranks/1', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.191 INFO:scrape: 2021-03-25T17:37:35.191 INFO:scrape:Failure: Test failure: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2021-03-25T17:37:35.191 INFO:scrape:1 jobs: ['5995383'] 2021-03-25T17:37:35.191 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/damage}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.191 INFO:scrape: 2021-03-25T17:37:35.191 INFO:scrape:Failure: Test failure: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2021-03-25T17:37:35.191 INFO:scrape:1 jobs: ['5995349'] 2021-03-25T17:37:35.191 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/alternate-pool}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.191 INFO:scrape: 2021-03-25T17:37:35.191 INFO:scrape:Failure: Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2021-03-25T17:37:35.192 INFO:scrape:1 jobs: ['5995307'] 2021-03-25T17:37:35.192 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.192 INFO:scrape: 2021-03-25T17:37:35.192 INFO:scrape:Dead: b'socket.timeout' 2021-03-25T17:37:35.192 INFO:scrape:1 jobs: ['5995421'] 2021-03-25T17:37:35.192 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.192 INFO:scrape: 2021-03-25T17:37:35.192 INFO:scrape:Failure: Test failure: test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) 2021-03-25T17:37:35.192 INFO:scrape:1 jobs: ['5995375'] 2021-03-25T17:37:35.192 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/client-recovery}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.193 INFO:scrape: 2021-03-25T17:37:35.193 INFO:scrape:Failure: Test failure: test_dir_deletion (tasks.cephfs.test_strays.TestStrays), test_dir_deletion (tasks.cephfs.test_strays.TestStrays) 2021-03-25T17:37:35.193 INFO:scrape:1 jobs: ['5995327'] 2021-03-25T17:37:35.193 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/strays}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.193 INFO:scrape: 2021-03-25T17:37:35.193 INFO:scrape:Dead: b'2021-03-25T17:07:03.296 DEBUG:teuthology.exit:Finished running handlers' 2021-03-25T17:37:35.193 INFO:scrape:1 jobs: ['5995371'] 2021-03-25T17:37:35.193 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/ubuntu_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-25T17:37:35.193 INFO:scrape: 2021-03-25T17:37:35.202 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/ failed: 61 dead: 2 running: 1 waiting: 0 queued: 0 passed: 91 Fail ================================================================= [5995251] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:15:11 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995251/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995251/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5e37c6dc591142a684496be583e54021 Command failed (workunit test suites/iozone.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5995253] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:14:46 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995253/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995253/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da396f705f784486901819b8800e9f16 Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull) [5995255] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995255/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995255/ sentry: https://sentry.ceph.com/organizations/ceph/?query=46eabb3616d1427d89d9b849e71dbbd3 'NoneType' object has no attribute 'mon_manager' [5995271] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} ----------------------------------------------------------------- time: 00:18:18 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995271/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995271/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54767e6ed3da47b2a685f8efbf20b63e Test failure: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) [5995273] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995273/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995273/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be0d11926e1b4d7cb3c06ac65ec1a0f0 Command failed on smithi151 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995281] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} ----------------------------------------------------------------- time: 00:16:17 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995281/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995281/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cb9c253c54f14923a383f5881b2e2bc6 Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) [5995283] fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:16:27 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995283/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995283/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8260ae78e0c44443946b09702a8ec6ca Command failed (workunit test suites/fsstress.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5995285] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:26:16 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995285/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995285/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a31f265390e841d2ab40593ca234fc52 Command failed on smithi042 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' [5995287] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:19:52 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995287/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995287/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a9b3a607f7214e06b854b0e756866221 Command failed (workunit test suites/dbench.sh) on smithi091 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh' [5995293] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:20:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995293/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995293/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5c9e2891743f4b878ab135179ed37c44 Test failure: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) [5995296] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:19:43 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995296/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995296/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b264abd44d448739768e049f8ba74eb 'NoneType' object has no attribute 'mon_manager' [5995297] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995297/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995297/ sentry: https://sentry.ceph.com/organizations/ceph/?query=42b69c82d9354ff39aefcc07f70a323d Command failed on smithi115 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995301] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 01:33:49 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995301/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995301/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5c8e9eee198746169179339db09c0ec2 Command failed on smithi146 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995302] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} ----------------------------------------------------------------- time: 00:18:28 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995302/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995302/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4377ec2b807641c1a738dff4112cd5ee Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks), test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) [5995303] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995303/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995303/ sentry: https://sentry.ceph.com/organizations/ceph/?query=90dec130bbad40afa7b8512d61c61fbc Command failed on smithi078 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995305] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} ----------------------------------------------------------------- time: 00:48:51 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995305/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995305/ sentry: https://sentry.ceph.com/organizations/ceph/?query=af7881da26674860ac8eeca4a6593c52 Test failure: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) [5995307] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995307/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995307/ sentry: https://sentry.ceph.com/organizations/ceph/?query=57c080f889b3479dbd515bbb5a2d694c Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) [5995308] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995308/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37f897caefcd43868ea85ec035d8d273 Command failed (workunit test kernel_untar_build.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5995327] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} ----------------------------------------------------------------- time: 00:19:38 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995327/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995327/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9650a33dadb44bf9b97a120bd381e16e Test failure: test_dir_deletion (tasks.cephfs.test_strays.TestStrays), test_dir_deletion (tasks.cephfs.test_strays.TestStrays) [5995331] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:16:45 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995331/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995331/ sentry: https://sentry.ceph.com/organizations/ceph/?query=590fef332c044be9b36058a724ed5d5e Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration) [5995332] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:17:17 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995332/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995332/ sentry: https://sentry.ceph.com/organizations/ceph/?query=556db43b1ca44aa49dabd315dae93d41 Command failed on smithi112 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995335] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} ----------------------------------------------------------------- time: 00:11:49 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995335/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995335/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c2f515ffcd664eec84beb15d2f403190 Command failed (workunit test fs/quota/quota.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/quota/quota.sh' [5995337] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995337/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995337/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d466bf72f29744e1ac9ccc57ead2c130 Command failed on smithi051 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995342] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:25:30 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995342/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995342/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c261b26083ae429b82e75192d240e272 Command failed on smithi002 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' [5995344] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} ----------------------------------------------------------------- time: 00:29:34 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995344/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995344/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5ab450cb8473447abe5eba7cc56c26c4 Test failure: test_add_data_pool_non_alphamueric_name_as_subdir (tasks.cephfs.test_admin.TestAdminCommands) [5995345] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 00:18:58 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995345/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995345/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fd15bfd29d2417c874371730722c876 Command failed (workunit test suites/dbench.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh' [5995348] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:17:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995348/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995348/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed8e3911793148949e23734fdf3f07b8 Command failed on smithi008 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995349] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} ----------------------------------------------------------------- time: 00:16:51 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995349/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995349/ sentry: https://sentry.ceph.com/organizations/ceph/?query=508bed08cbb74aab92e3e0d7a8d5ab50 Test failure: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) [5995353] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:26:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995353/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995353/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4893557d61f04764ab635f5dd84deb85 Command failed on smithi151 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995355] fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995355/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995355/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d8c566680e744094ab7c2725913dd111 Command failed (workunit test suites/fsstress.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5995358] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995358/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995358/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6cd2bf5a8e64da689254924c7b0995e Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi073 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5995361] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} ----------------------------------------------------------------- time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995361/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995361/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b142cd8e1874aeda9998aff40f6c9e2 Test failure: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) [5995363] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995363/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995363/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9850744e1f4748f8b77822e3bb020524 'NoneType' object has no attribute 'mon_manager' [5995364] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} ----------------------------------------------------------------- time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995364/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995364/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a85ae26804fb43eea5ac9bae62ed8b7f Test failure: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) [5995365] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:25:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995365/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995365/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c50f42c0452d4c9d8d5f348ba04ba0b1 Command failed on smithi150 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' [5995366] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:17:48 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995366/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995366/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6564ae02b1254848b5bf1aff992ffebe Test failure: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [5995367] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995367/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995367/ "2021-03-25T05:14:45.545810+0000 mds.a (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000003b1 (/client.0/tmp/fsstress/ltp- full-20091231/testcases/commands) see mds.a log and `damage ls` output for details" in cluster log [5995374] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:44:09 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995374/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995374/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ec0e05d02d6437ab8d5de73481d059b Command failed on smithi063 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995375] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:23:22 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995375/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995375/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fbc5a444ebe5476eb74ea27804d40fa9 Test failure: test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) [5995378] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:33:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995378/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995378/ sentry: https://sentry.ceph.com/organizations/ceph/?query=355b73a8c40f4c6694bd22ba1948424e Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) [5995380] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:16:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995380/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995380/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14b14b828d7e43af8e3b927e39c72943 Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5995382] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 01:08:10 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995382/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995382/ sentry: https://sentry.ceph.com/organizations/ceph/?query=969137ab18624924b2a361dad5d77fc5 Command failed on smithi086 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995383] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} ----------------------------------------------------------------- time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995383/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995383/ sentry: https://sentry.ceph.com/organizations/ceph/?query=93de165e65ca4841b26a6dd85f0c7a37 Test failure: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) [5995387] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:19:41 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995387/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995387/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9086a2f24d7740b5a26dc923b347c7a1 'NoneType' object has no attribute 'mon_manager' [5995388] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} ----------------------------------------------------------------- time: 00:17:23 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995388/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995388/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eb2ecccfd5724cebaf98414b9ac65cbc Command failed on smithi111 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995389] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:33:27 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995389/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995389/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fcf97b00fc60409392c7ed9a288f07ec Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) [5995391] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:23:34 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995391/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995391/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ffbc65671a2498f9e00f4ceb22d7572 Command failed on smithi038 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' [5995392] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} ----------------------------------------------------------------- time: 00:49:41 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995392/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995392/ sentry: https://sentry.ceph.com/organizations/ceph/?query=601eabf82440403e8cb1d4257afae987 Test failure: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) [5995398] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} ----------------------------------------------------------------- time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995398/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995398/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7557d24b78464c16b94b83d09acd189b Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation) [5995399] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} ----------------------------------------------------------------- time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995399/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995399/ sentry: https://sentry.ceph.com/organizations/ceph/?query=18c61e8b27c543a19a37de4d35d40cef Command failed (workunit test fs/test_o_trunc.sh) on smithi073 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh' [5995402] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:31:03 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995402/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995402/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b6edcdde57d4b148679252fd1f8adb2 Command failed on smithi036 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995404] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:48:17 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995404/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995404/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8eb6ef030239416fbfbf37588b6cb5a2 Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) [5995405] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:14:34 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995405/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995405/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6ba591f65a964bb6a2c0a68074c01ecd Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi074 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5995410] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995410/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995410/ sentry: https://sentry.ceph.com/organizations/ceph/?query=731f560b86944403b133e8c5f2b8ed4e Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull) [5995412] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995412/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995412/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9571834ea35f4cdba7888502bf360584 Command failed on smithi031 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995413] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} ----------------------------------------------------------------- time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995413/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995413/ "2021-03-25T05:49:37.283862+0000 mds.d (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi117:0 (4746), after 303.606 seconds" in cluster log [5995414] fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:31:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995414/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995414/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c3e07c235404d2ea158677def3b721a Command failed (workunit test suites/fsstress.sh) on smithi151 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3a37f398e9d4da4d532fb63c7eb9ff22bb0a35a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5995418] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:18:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995418/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995418/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9dc3b376e6d64aab81208c22ab17b1fc Command failed on smithi150 with status 1: 'mkdir -- /home/ubuntu/cephtest/mnt.0' [5995424] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 02:34:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995424/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995424/ reached maximum tries (150) after waiting for 900 seconds [5995425] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} ----------------------------------------------------------------- time: 00:18:35 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995425/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995425/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1eec2b2fa63c4e5cb41ef05f42bf69e2 Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) [5995426] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995426/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995426/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1194b3bd2fa741339b39d12c99428a05 Command failed on smithi134 with status 1: 'cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0' Dead ================================================================= [5995330] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995330/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995330/ [5995371] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995371/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995371/ hit max job timeout Running ================================================================= [5995421] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995421/ Pass ================================================================= [5995249] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:20:14 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995249/ [5995257] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995257/ [5995259] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:10:03 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995259/ [5995261] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:15 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995261/ [5995263] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:09:59 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995263/ [5995265] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995265/ [5995267] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995267/ [5995269] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995269/ [5995275] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:29:58 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995275/ [5995277] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:36:03 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995277/ [5995279] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:13:21 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995279/ [5995289] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995289/ [5995291] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995291/ [5995295] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:11:30 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995295/ [5995298] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995298/ [5995299] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:28 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995299/ [5995300] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995300/ [5995304] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:54:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995304/ [5995306] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995306/ [5995309] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:20:50 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995309/ [5995310] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:20:01 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995310/ [5995311] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:14:13 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995311/ [5995312] fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073} time: 00:08:10 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995312/ [5995313] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} time: 00:43:28 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995313/ [5995314] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:08:27 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995314/ [5995315] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 00:56:33 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995315/ [5995316] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:33:01 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995316/ [5995317] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:47 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995317/ [5995318] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:30:30 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995318/ [5995319] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:19 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995319/ [5995320] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:33:50 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995320/ [5995321] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:12:00 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995321/ [5995322] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{centos_8} tasks/fstop} time: 00:09:20 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995322/ [5995323] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:19:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995323/ [5995324] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:58 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995324/ [5995325] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 01:15:28 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995325/ [5995326] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:29:33 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995326/ [5995328] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 03:20:36 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995328/ [5995329] fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:53:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995329/ [5995333] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:11:57 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995333/ [5995334] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995334/ [5995336] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:10:48 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995336/ [5995338] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:12:37 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995338/ [5995339] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995339/ [5995340] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:56 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995340/ [5995341] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} time: 01:12:42 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995341/ [5995343] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:41:29 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995343/ [5995346] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:08:56 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995346/ [5995347] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995347/ [5995350] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:38:58 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995350/ [5995351] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:11:11 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995351/ [5995352] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:47:56 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995352/ [5995354] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:20:31 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995354/ [5995356] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:12:14 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995356/ [5995357] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:13:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995357/ [5995359] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995359/ [5995360] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:09:44 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995360/ [5995362] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:34:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995362/ [5995368] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:18:41 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995368/ [5995369] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:13:28 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995369/ [5995370] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:11:15 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995370/ [5995372] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995372/ [5995373] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995373/ [5995376] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:23:12 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995376/ [5995377] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995377/ [5995379] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995379/ [5995381] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:41:21 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995381/ [5995384] fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} time: 00:52:55 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995384/ [5995385] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:19:26 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995385/ [5995386] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:27:15 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995386/ [5995390] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:39:40 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995390/ [5995393] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:11:02 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995393/ [5995394] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:14 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995394/ [5995395] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 02:25:51 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995395/ [5995396] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:19:05 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995396/ [5995397] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:40 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995397/ [5995400] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995400/ [5995401] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:15:52 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995401/ [5995403] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995403/ [5995406] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:43 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995406/ [5995407] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:56 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995407/ [5995408] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:25:07 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995408/ [5995409] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995409/ [5995411] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:20:45 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995411/ [5995415] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995415/ [5995416] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:34 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995416/ [5995417] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:17:27 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995417/ [5995419] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995419/ [5995420] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:51 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995420/ [5995422] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:08:55 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995422/ [5995423] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:16:52 info: http://pulpito.ceph.com/yuriw-2021-03-24_22:19:04-fs-wip-yuri-testing-2021-03-24-0910-pacific-distro-basic-smithi/5995423/