List view
For `type: pass-fail` - [ ] Apply `{input, output}_validator_flags` from `testdata.yaml` to the respective testcase when validating - [ ] Collect testcase verdicts, infer testdata verdicts for each testgroup - [ ] Update UI to report testgroup verdicts For `type: scoring`: - [ ] Allow testcase verdicts to have numerical `score` - [ ] Apply `grading` flags from `testdata.yaml` to respective testcase scores - [ ] Apply aggregation rules from `grading` in `testdata.yaml` - [ ] Update UI to report aggregate scores for testgroups Expectations: - [ ] Read and parse `expectations.yaml` - [ ] Compare testcase verdicts with expectations - [ ] Compare testgroup verdicts with expectations
No due date•3/4 issues closed