2023-09-21T01:06:31.951 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T01:06:31.952 INFO:teuthology.results:seed: '5790' 2023-09-21T01:07:21.394 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T01:07:21.401 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-09-21T13:08:16.075 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-09-21T13:08:16.290 INFO:scrape:Found 1 jobs 2023-09-21T13:08:17.023 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.024 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.024 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.025 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.025 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.026 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.026 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.026 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.027 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.028 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.028 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.029 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.029 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.030 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.030 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.031 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.031 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.032 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.032 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.032 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.033 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.033 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.034 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.034 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.035 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.035 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.036 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.036 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.037 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.037 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.038 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.038 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.039 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.039 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.040 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.040 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.041 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.041 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.042 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.042 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.043 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.043 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.044 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.044 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.044 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.045 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.045 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.046 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.046 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.047 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.047 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.048 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.048 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.049 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.049 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.050 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.050 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.051 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.051 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.052 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.052 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.053 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.053 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.053 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.054 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.055 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.055 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.055 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.056 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.056 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.057 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.057 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.058 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.058 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.059 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.059 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.060 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.060 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.061 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.061 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.062 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.062 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.063 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.063 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.063 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.064 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.064 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.065 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.065 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.066 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.066 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.067 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.067 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.068 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.068 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.069 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.069 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.070 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.070 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.070 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.071 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.071 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.072 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.072 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.073 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.073 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.074 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.074 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.075 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.075 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.076 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.076 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.077 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.077 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.078 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.078 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.079 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.079 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.079 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.080 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.080 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.081 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.081 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.082 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.082 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.083 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.083 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.084 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.084 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.085 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.085 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.086 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.086 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.087 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.087 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.087 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.088 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.088 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.089 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.089 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.090 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.090 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.091 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.091 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.092 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.092 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.093 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.093 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.094 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.094 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.095 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.095 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.096 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.096 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.096 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.097 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.097 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.098 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.098 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.099 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.099 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.100 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.100 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.101 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.101 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.101 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.102 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.102 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.103 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.103 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.104 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.104 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.105 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.105 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.106 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.106 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.107 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.107 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.108 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.108 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.109 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.109 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.110 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.110 WARNING:scrape:Saw end of BT but not start 2023-09-21T13:08:17.223 INFO:scrape:Found 1 distinct failure reasons 2023-09-21T13:08:17.224 INFO:scrape:Dead: b'2023-09-21T13:08:14.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 13:08:14 smithi119 ceph-mon[99159]: pgmap v20721: 97 pgs: 97 active+clean; 21 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s' 2023-09-21T13:08:17.224 INFO:scrape:1 jobs: ['7399948'] 2023-09-21T13:08:17.225 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs:workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'n/5', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-no-extra-options}'] 2023-09-21T13:08:17.225 INFO:scrape: 2023-09-21T13:08:17.226 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2023-09-21_01:05:43-fs:workload-wip-vshankar-testing-20230824.045828-scrub-failure-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2023-09-21_01:05:43-fs:workload-wip-vshankar-testing-20230824.045828-scrub-failure-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2023-09-21_01:05:43-fs:workload-wip-vshankar-testing-20230824.045828-scrub-failure-testing-default-smithi/ failed: 0 dead: 0 running: 1 waiting: 0 queued: 0 passed: 0 Running ================================================================= [7399948] fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} info: http://pulpito.ceph.com/mchangir-2023-09-21_01:05:43-fs:workload-wip-vshankar-testing-20230824.045828-scrub-failure-testing-default-smithi/7399948/