Add Span Level Evals (multi-step evaluation)#32

Merged

JCamyre merged 22 commits intomainfrom

joseph/span-level-evals

Jan 7, 2025

Commits on Jan 5, 2025

Add function declaration for an async run evaluation function for the JudgmentClient.
JCamyre
committed
Add sample span-level evaluation call to a traced function.
JCamyre
committed
Add function to asynchronously evaluate on the span-level, calling JudgmentClient's run eval function.
JCamyre
committed
Clean up Tracer's __init__ function. Add function to return the current TraceClient (to allow users to evaluate on span-level).
JCamyre
committed
Add a better evaluation for testing. Fix tracer_id missing issue by calling trace.save() at the start of trace (will hide this). Handle asynchronous functions.
JCamyre
committed
Add evaluation_result to TraceEntry.
JCamyre
committed
Fix some run eval logic. Add logic to record Evaluation results.
JCamyre
committed
Add temp fix to handle out of order entries (due to async evaluations). Properly convert entries to strings to save evaluation_result's.
JCamyre
committed

Commits on Jan 6, 2025

Commits on Jan 7, 2025