Skip to content

feat: add third-party evaluation metrics #661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 25 commits into from
Apr 2, 2025
Merged

feat: add third-party evaluation metrics #661

merged 25 commits into from
Apr 2, 2025

Conversation

cameronraysmith
Copy link
Collaborator

@cameronraysmith
fix(metrics): add external trajectory metrics
325169f 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith cameronraysmith self-assigned this Apr 1, 2025
@cameronraysmith
fix(tasks): add evaluation task
0f1cc42 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(tasks): export evaluation task
21d3e18 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): add trajectory evaluation outputs
1c80ccb 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): add task to evaluate trajectory metrics
f965992 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith cameronraysmith added skip-ci Skip CI workflow build-images Control image build on PRs dev-mode Do not rebuild images and use branch name instead of git short SHA for image tags labels Apr 1, 2025
@cameronraysmith cameronraysmith added this to the 0.4.x milestone Apr 1, 2025
@cameronraysmith cameronraysmith removed the skip-ci Skip CI workflow label Apr 1, 2025
@cameronraysmith
test(metrics): add smoke tests for trajectory module
49e0a76 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(metrics): handle sparse and dense neighbor graph data
9edb447 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
test(tasks): add smoke tests for evaluate module
6bb0eba 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(evaluate): accept in-memory objects in addition to file paths
832e359 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(evaluate): perform all dataframe manipulation outside plot helper
f86f056 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(utils): refactor string diff line endings
912b107 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(constants): presume developmental cluster orderings
ff996fc 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(constants): map model identifiers to velocity layer keys
71032aa 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(evaluate): decompose calculate_cross_boundary_correctness
08f7044 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
test(tasks): update evaluate module tests
d03e4dc 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): use evaluate_trajectory_metrics task
bb99a9e 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith cameronraysmith added skip-ci Skip CI workflow execute-workflow Control workflow execution on PRs and removed build-images Control image build on PRs skip-ci Skip CI workflow labels Apr 2, 2025
@cameronraysmith
fix(workflows): increase memory limit for evaluate trajectory metrics
65890b2 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(tasks): set evaluation data set and model ordering
9487e4d 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(constants): update model ordering
fde8c65 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): set time lineage fate correlation upload path
3c0fc4a 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): set trajectory metrics upload path
c41bca4 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
chore(workflows): bump trajectory evaluation cache 2024.8.15.1
f58f365 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(tasks): update time fate correlation output directory
c9d90b4 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith
fix(workflows): update combine time fate correlation task output dire…
a1d3429 
…ctory

Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith cameronraysmith added skip-ci Skip CI workflow and removed execute-workflow Control workflow execution on PRs dev-mode Do not rebuild images and use branch name instead of git short SHA for image tags labels Apr 2, 2025
@cameronraysmith
chore(version): 0.4.2
66cbf09 
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
@cameronraysmith cameronraysmith marked this pull request as ready for review April 2, 2025 18:10
@cameronraysmith cameronraysmith requested a review from a team as a code owner April 2, 2025 18:10
@cameronraysmith cameronraysmith added skip-ci Skip CI workflow enhancement New feature or request python Pull requests that update Python code and removed skip-ci Skip CI workflow labels Apr 2, 2025
@cameronraysmith cameronraysmith merged commit 66cbf09 into main Apr 2, 2025
69 checks passed
@cameronraysmith cameronraysmith deleted the 660-eval branch April 2, 2025 18:18
@cameronraysmith cameronraysmith mentioned this pull request Apr 8, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@cameronraysmith cameronraysmith

Labels
enhancement New feature or request python Pull requests that update Python code
Projects
None yet
Milestone
0.4.x
Development

Successfully merging this pull request may close these issues.

add third-party evaluation metrics
1 participant
@cameronraysmith