2023-10-26T09:03:52.750 INFO:root:teuthology version: 0.0.1.dev201+g202b180
2023-10-26T09:03:52.751 INFO:teuthology.results:subset: '111/120000'
2023-10-26T09:03:52.751 INFO:teuthology.results:seed: '7106'
2023-10-26T09:40:36.739 INFO:root:teuthology version: 0.0.1.dev201+g202b180
2023-10-26T09:40:36.774 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-10-26T11:48:46.617 INFO:teuthology.results:Tests finished! gathering results...
2023-10-26T11:48:47.684 INFO:scrape:Found 14 jobs
2023-10-26T11:48:49.006 WARNING:scrape:Misunderstood line: 
2023-10-26T11:48:51.413 WARNING:scrape:Misunderstood line: 
2023-10-26T11:48:56.109 INFO:scrape:Found 8 distinct failure reasons
2023-10-26T11:48:56.110 INFO:scrape:Failure: Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)
2023-10-26T11:48:56.110 INFO:scrape:1 jobs: ['7437166']
2023-10-26T11:48:56.111 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/v1', 'osd}', 'overrides/{distro/testing/k-testing}}', 'overrides/{ignorelist_health', 'pg-warn}', 'tasks/fscrypt-common}']
2023-10-26T11:48:56.111 INFO:scrape:
2023-10-26T11:48:56.112 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi131 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e652850897f60c32cb9cc05022cf9f93e3b00db1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh'
2023-10-26T11:48:56.112 INFO:scrape:2 jobs: ['7437190', '7437178']
2023-10-26T11:48:56.112 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}']
2023-10-26T11:48:56.113 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}']
2023-10-26T11:48:56.113 INFO:scrape:
2023-10-26T11:48:56.113 INFO:scrape:Failure: Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS)
2023-10-26T11:48:56.114 INFO:scrape:1 jobs: ['7437170']
2023-10-26T11:48:56.114 INFO:scrape:suites: ['fs/nfs/{cluster/{1-node}', 'overrides/ignorelist_health', 'supported-random-distros$/{ubuntu_20.04}', 'tasks/nfs}']
2023-10-26T11:48:56.115 INFO:scrape:
2023-10-26T11:48:56.115 INFO:scrape:Valgrind: mon (InvalidRead, Leak_PossiblyLost, Leak_StillReachable)
2023-10-26T11:48:56.115 INFO:scrape:1 jobs: ['7437192']
2023-10-26T11:48:56.116 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'ranks/1', 'session_timeout}', 'tasks/fsstress', 'validater/valgrind}']
2023-10-26T11:48:56.116 INFO:scrape:
2023-10-26T11:48:56.117 INFO:scrape:Failure: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)
2023-10-26T11:48:56.117 INFO:scrape:1 jobs: ['7437168']
2023-10-26T11:48:56.117 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/client-limits}']
2023-10-26T11:48:56.118 INFO:scrape:
2023-10-26T11:48:56.118 INFO:scrape:Failure: error during scrub thrashing: rank damage found: {'backtrace'}
2023-10-26T11:48:56.119 INFO:scrape:2 jobs: ['7437184', '7437186']
2023-10-26T11:48:56.119 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}']
2023-10-26T11:48:56.119 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/fs/misc}}', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/multi/{export-check', 'replication/always}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}']
2023-10-26T11:48:56.120 INFO:scrape:
2023-10-26T11:48:56.120 INFO:scrape:Failure: Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring)
2023-10-26T11:48:56.121 INFO:scrape:1 jobs: ['7437172']
2023-10-26T11:48:56.121 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}']
2023-10-26T11:48:56.121 INFO:scrape:
2023-10-26T11:48:56.122 INFO:scrape:Failure: Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)
2023-10-26T11:48:56.122 INFO:scrape:1 jobs: ['7437188']
2023-10-26T11:48:56.122 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/strays}']
2023-10-26T11:48:56.123 INFO:scrape: