Skip to content

DON'T MERGE YET: update: API calls to pass API key via auth headers (All routes including evaluation routes) #78

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 28, 2025

Conversation

jack-devhub
Copy link
Contributor

@jack-devhub jack-devhub commented Feb 21, 2025

📝 Implementation Details

Key Components:

  1. Header-Based Authentication Implementation
# Common pattern across all API calls
headers={
    "Content-Type": "application/json",
    "Authorization": f"Bearer {self.judgment_api_key}"
}
  1. API Key Removal from Request Bodies
# Before: API key in request body
json={"api_key": self.judgment_api_key}

# After: API key removed
json={}  # Empty body with auth header
  1. Updated Endpoint Security Across All Routes
# Dataset Client Before
response = requests.post(URL, json={"judgment_api_key": key})

# Dataset Client After 
response = requests.post(URL, json=content, headers=auth_headers)

✅ Requirement Validation

Task Requirement Implementation Status Test Coverage
1. Move API keys to headers ✅ All services updated Existing tests modified
2. Remove body API keys ✅ All relevant models updated Full coverage
3. Cover evaluation routes evaluation.py updated ✅ New tests added

🧪 Test Updates

Modified Test File: src/e2etests/test_tracer.py
Newly Updated: src/judgeval/evaluation.py
Key Changes:

# Added async test support
@pytest.mark.asyncio
async def test_evaluation_mixed(test_input):
    # Uses fixture with header-based auth
    upper = await make_upper(test_input)

# New test input fixture  
@pytest.fixture
def test_input():
    return "What if these shoes don't fit?"

Key Validations

  1. Auth Header Propagation

    • Verified all API calls (including evaluation routes) use header auth
    • Confirmed dataset push/pull operations authenticate via headers
  2. Backward Compatibility

    • Maintained existing test structures with auth upgrades
    • Preserved async test functionality with pytest markers

Implementation Notes

  1. Bearer Token Standardization
    • Consistent use of Authorization: Bearer <key> format across all routes
  2. Exception Safety
    • Maintained existing error handling patterns

Final Updates:

  • Now includes evaluation routes, ensuring all endpoints comply with header-based authentication
  • Fully validated with updated tests

@JCamyre JCamyre changed the title update: API calls to pass API key via auth headers (excluding evaluation routes) DON'T MERGE YET: update: API calls to pass API key via auth headers (excluding evaluation routes) Feb 23, 2025
@jack-devhub jack-devhub changed the title DON'T MERGE YET: update: API calls to pass API key via auth headers (excluding evaluation routes) DON'T MERGE YET: update: API calls to pass API key via auth headers (All routes including evaluation routes) Feb 25, 2025
- Modify all API calls in judgment_client, run_evaluation, and tracer to use Authorization header
- Update e2etests to reflect new authentication method
- Remove explicit judgment_api_key from JSON payload in most API calls
- Adjust test cases to expect different error codes for missing authentication
@JCamyre JCamyre changed the base branch from main to az-update-headers February 28, 2025 06:22
@JCamyre JCamyre changed the base branch from az-update-headers to main February 28, 2025 06:22
Copy link
Collaborator

@JCamyre JCamyre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JCamyre JCamyre merged commit bb3aa11 into main Feb 28, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants