2023-11-27T15:51:48.813 INFO:root:teuthology version: 0.0.1.dev205+g2442239
2023-11-27T15:51:48.813 INFO:teuthology.results:seed: '3775'
2023-11-27T15:53:08.130 INFO:root:teuthology version: 0.0.1.dev205+g2442239
2023-11-27T15:53:08.134 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-28T03:54:04.166 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-11-28T03:54:05.192 INFO:scrape:Found 10 jobs
2023-11-28T03:54:05.469 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.470 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.470 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.470 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.590 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.590 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.591 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.591 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.695 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.696 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.696 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:05.696 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.621 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.622 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:07.623 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:08.955 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:08.956 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:08.956 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:08.957 WARNING:scrape:Saw end of BT but not start
2023-11-28T03:54:10.041 INFO:scrape:Found 3 distinct failure reasons
2023-11-28T03:54:10.042 INFO:scrape:Failure: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a64124f0d915de37f45eaaa6de792b4da0f32dcf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh'
2023-11-28T03:54:10.042 INFO:scrape:5 jobs: ['7467823', '7467817', '7467820', '7467819', '7467818']
2023-11-28T03:54:10.042 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'session_timeout}', 'standby-replay']
2023-11-28T03:54:10.042 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '2-scrub/yes', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/legacy', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'ranks/1', 'ranks/multi/{export-check', 'replication/always}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'tasks/{0-subvolume/{with-no-extra-options}', 'tasks/{0-subvolume/{with-quota}', 'wsync/no}', 'wsync/yes}']
2023-11-28T03:54:10.042 INFO:scrape:
2023-11-28T03:54:10.043 INFO:scrape:Failure: error during scrub thrashing: rank damage found: {'backtrace'}
2023-11-28T03:54:10.043 INFO:scrape:1 jobs: ['7467822']
2023-11-28T03:54:10.043 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{no-subvolume}', 'wsync/yes}']
2023-11-28T03:54:10.043 INFO:scrape:
2023-11-28T03:54:10.043 INFO:scrape:Dead: b'2023-11-28T03:54:08.394 INFO:journalctl@ceph.mon.b.smithi093.stdout:Nov 28 03:54:07 smithi093 ceph-mon[100178]: osdmap e1460: 12 total, 12 up, 12 in'
2023-11-28T03:54:10.043 INFO:scrape:1 jobs: ['7467815']
2023-11-28T03:54:10.043 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/3', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/multi/{export-check', 'replication/always}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'wsync/no}']
2023-11-28T03:54:10.043 INFO:scrape: