Skip to content

Staging -> Prod #303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jun 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
4037f73
Staging and auto package release (#275)
justinsheu May 22, 2025
5976dd4
Trace Usage Edits (#266)
alanzhang25 May 22, 2025
3ec8d9b
update litellm pyproject and fix e2etest (#276)
Mandolaro May 23, 2025
2186195
Agent Names added (#270)
adivate2021 May 23, 2025
ebbc5d5
Fix assert test e2e test (#277)
justinsheu May 23, 2025
3a2d222
add evaluation link (#279)
Mandolaro May 24, 2025
6e45521
Image file updates and social media badges (#274)
shunuen0 May 24, 2025
84be26b
Testing updates (#269)
justinsheu May 24, 2025
52ef0d9
add new trace column (#281)
Mandolaro May 25, 2025
b8f97f1
change timeout to use variable (#282)
justinsheu May 25, 2025
62cd759
add code coverage for staging e2es (#285)
justinsheu May 28, 2025
9f40a4d
Add ToolDependency Scorer and Parameter Checking (#253)
adivate2021 May 28, 2025
f145d60
modified deep tracing to chain with existing tracefuncs (#286)
justinsheu May 29, 2025
aaf6d27
Refactor agent names (#287)
adivate2021 May 29, 2025
ce99764
Handle un-serializable types in Traces. Filter 'self' arguments. (#278)
JCamyre May 29, 2025
0f0e337
Example Created At (#284)
alanzhang25 May 29, 2025
f6652d3
fix logging for assert test trace (#290)
Mandolaro May 30, 2025
5b8c1d9
E2E additions based on assigned judgeval files (#283)
justinsheu May 30, 2025
fa1bfe3
Merge branch 'main' into staging
justinsheu May 30, 2025
1ca8086
Merge branch 'main' into staging
justinsheu May 31, 2025
65dc640
Make default project name uuid (#293)
rishi763 May 31, 2025
d8d2491
Raise warning for missing Org ID or API Key (#294)
Mandolaro Jun 1, 2025
564723e
Agent state (#295)
andrewli2403 Jun 2, 2025
4358fad
Update Docs link (#297)
JCamyre Jun 2, 2025
6067b28
Ahh/http error recording (#289)
abhishekg999 Jun 2, 2025
9e18453
Update README.md (#298)
JCamyre Jun 3, 2025
cdc2e4a
add workflow (#300)
justinsheu Jun 3, 2025
30addd8
Example Check and User Prompt (#301)
alanzhang25 Jun 3, 2025
c81cd95
Trace Datasets (#291)
alanzhang25 Jun 3, 2025
ee98444
Update blocked-pr.yaml (#302)
justinsheu Jun 3, 2025
c501733
removed src/demo (#299)
justinsheu Jun 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/blocked-pr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Check Blocked PR

on:
pull_request:
types:
- opened
- labeled
- unlabeled
- synchronize

jobs:
fail-for-blocked:
if: contains(github.event.pull_request.labels.*.name, 'Blocked')
runs-on: ubuntu-latest
steps:
- name: Fail if PR is blocked
run: |
echo "This PR is currently blocked. Please unblock it before merging."
exit 1
39 changes: 24 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@

<br>

## [🌐 Landing Page](https://www.judgmentlabs.ai/) • [📚 Docs](https://judgment.mintlify.app/getting_started) • [🚀 Demos](https://www.youtube.com/@AlexShan-j3o)
## [🌐 Landing Page](https://www.judgmentlabs.ai/) • [📚 Docs](https://docs.judgmentlabs.ai/introduction) • [🚀 Demos](https://www.youtube.com/@AlexShan-j3o)

[![X](https://img.shields.io/badge/-X/Twitter-000?logo=x&logoColor=white)](https://x.com/JudgmentLabs)
[![LinkedIn](https://custom-icon-badges.demolab.com/badge/LinkedIn%20-0A66C2?logo=linkedin-white&logoColor=fff)](https://www.linkedin.com/company/judgmentlabs)
[![Discord](https://img.shields.io/badge/-Discord-5865F2?logo=discord&logoColor=white)](https://discord.gg/FMxHkYTtFE)
[![Discord](https://img.shields.io/badge/-Discord-5865F2?logo=discord&logoColor=white)](https://discord.gg/ZCnSXYug)

</div>

Expand All @@ -28,19 +28,28 @@ We support tracing agents built with LangGraph, OpenAI SDK, Anthropic, ... and a
Judgeval is created and maintained by [Judgment Labs](https://judgmentlabs.ai/).

## 📋 Table of Contents
* [✨ Features](#-features)
* [🔍 Tracing](#-tracing)
* [🧪 Evals](#-evals)
* [📡 Monitoring](#-monitoring)
* [📊 Datasets](#-datasets)
* [💡 Insights](#-insights)
* [🛠️ Installation](#️-installation)
* [🏁 Get Started](#-get-started)
* [🏢 Self-Hosting](#-self-hosting)
* [📚 Cookbooks](#-cookbooks)
* [💻 Development with Cursor](#-development-with-cursor)
* [⭐ Star Us on GitHub](#-star-us-on-github)
* [❤️ Contributors](#️-contributors)
- [🌐 Landing Page • 📚 Docs • 🚀 Demos](#-landing-page----docs---demos)
- [Judgeval: open-source testing, monitoring, and optimization for AI agents](#judgeval-open-source-testing-monitoring-and-optimization-for-ai-agents)
- [📋 Table of Contents](#-table-of-contents)
- [✨ Features](#-features)
- [🛠️ Installation](#️-installation)
- [🏁 Get Started](#-get-started)
- [🛰️ Tracing](#️-tracing)
- [📝 Offline Evaluations](#-offline-evaluations)
- [📡 Online Evaluations](#-online-evaluations)
- [🏢 Self-Hosting](#-self-hosting)
- [Key Features](#key-features)
- [Getting Started](#getting-started)
- [📚 Cookbooks](#-cookbooks)
- [Sample Agents](#sample-agents)
- [💰 LangGraph Financial QA Agent](#-langgraph-financial-qa-agent)
- [✈️ OpenAI Travel Agent](#️-openai-travel-agent)
- [Custom Evaluators](#custom-evaluators)
- [🔍 PII Detection](#-pii-detection)
- [📧 Cold Email Generation](#-cold-email-generation)
- [💻 Development with Cursor](#-development-with-cursor)
- [⭐ Star Us on GitHub](#-star-us-on-github)
- [❤️ Contributors](#️-contributors)

<!-- Created by https://github.com/ekalinin/github-markdown-toc -->

Expand Down
146 changes: 0 additions & 146 deletions src/demo/async_evaluation_example.py

This file was deleted.

Loading
Loading