-
Notifications
You must be signed in to change notification settings - Fork 82
Update README clustering pic to darkmode #262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…rove other sections with feedback from Aaryan.
…entLabs/judgeval into alex/update-readme-pics
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello @SecroLoL, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
Hello! Gemini or gemini-code-assist here, providing a summary of this pull request.
This PR focuses on updating the README.md file. The main goals, as stated in the title and description, are to update the clustering image to a dark mode version and make several wording adjustments. The changes involve a significant restructuring of the README content, introducing a new feature table with screenshots, adding a "Cookbooks" section, updating installation instructions, and refining existing text for clarity and accuracy.
Highlights
- README Restructuring: The layout and content organization of the
README.mdfile have been significantly updated. This includes a new header with icons, a Table of Contents, and a revised introduction. - Updated Features Section with Images: The previous bulleted list of features has been replaced with a new table format. This table provides a brief description and a corresponding screenshot for each core feature (Tracing, Evals, Monitoring, Datasets, Insights).
- Dark Mode Clustering Image: As per the PR title, the image used to showcase the "Insights" feature in the new table has been updated to a dark mode version (
assets/dataset_clustering_screenshot_dm.png). - Added Cookbooks Section: A new section titled "Cookbooks" has been added. This section highlights sample agents and custom evaluators available in the
judgment-cookbookrepository, providing quick links and descriptions. - Installation Instructions Update: The installation instructions have been updated to clarify that both
JUDGMENT_API_KEYandJUDGMENT_ORG_IDenvironment variables are required to connect to the Judgment platform. The link for creating an account has also been updated.
Changelog
Click here to see the changelog
- README.md
- Replaced header links with icons and added a Discord link.
- Added an image (
assets/experiments_page.png) below the main title. - Rewrote the introductory section about Judgeval.
- Added a Table of Contents.
- Replaced the old bulleted features list with a new table format including descriptions and screenshots for each feature.
- Updated the "Insights" feature image to
assets/dataset_clustering_screenshot_dm.png. - Updated installation instructions to include
JUDGMENT_ORG_IDand a new registration link. - Minor wording change in the "Online Evaluations" section.
- Updated code example in "Online Evaluations" from
judgment.get_current_trace().async_evaluatetojudgment.async_evaluate. - Added a new "Cookbooks" section with highlights from the
judgment-cookbookrepo. - Renamed "Contributing" to "❤️ Contributors" and added a contributors collage image.
- Removed the old "Documentation and Demos" section.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
README file updated,
Dark mode image now shines bright,
Words flow, clear and clean.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request significantly enhances the README.md file, making it more comprehensive, user-friendly, and visually appealing. Key improvements include the addition of a Table of Contents, a revamped "Features" section with illustrative images (including the updated dark mode clustering image), and a new "Cookbooks" section. The wording changes generally improve clarity, and the updates to API key information and example code are also noted. Overall, these are excellent changes that will greatly benefit users.
In the absence of a specific style guide for Markdown, my review focused on general best practices for README files, such as clarity, accuracy, up-to-date information, and maintainability of the Markdown.
Summary of Findings
- API Example Accuracy: The
async_evaluateAPI call in the "Online Evaluations" example (line 153) was simplified. It's important to confirm this change accurately reflects the library's current public API and intended usage. (Severity: Medium) - Discord Invite Link Stability: The Discord link
https://discord.gg/taAufyhfin the "Cookbooks" section (line 183) should be verified as a permanent invite to ensure long-term accessibility for the community. (Severity: Medium) - Image Usage Consistency: The image
assets/experiments_page.pngis used both as a general introductory image (line 8) and specifically for the "Evals" feature (line 47). This is a minor stylistic point regarding potential repetitiveness, but not a critical issue. (Severity: Low - Not commented due to review settings) - HTML in Markdown Tables: The "Features" section table (lines 44-50) uses HTML tags (
<h3>, imagewidth). While GFM renders this, it can reduce raw Markdown readability and maintainability compared to pure Markdown, though it offers more formatting control. This is a minor consideration. (Severity: Low - Not commented due to review settings) - Clarity of "Attach performance monitoring": The phrasing "Attach performance monitoring on traces" (line 129) is concise. It could be slightly reworded to more directly connect with "online evaluations" as shown in the example, e.g., "Measure system quality by running online evaluations within your traces." This is a minor wording suggestion. (Severity: Low - Not commented due to review settings)
Merge Readiness
This pull request introduces substantial and valuable improvements to the README. The new structure and content significantly enhance its utility.
Before merging, I recommend confirming the two medium-severity points raised:
- The correctness of the updated
async_evaluateAPI example. - The stability and permanence of the Discord invite link.
Once these points are addressed or confirmed, the README will be in excellent shape. As a reviewer, I am not authorized to approve pull requests; please ensure further review and approval as per your team's process.
README.md
Outdated
| @@ -142,7 +150,7 @@ def main(): | |||
| messages=[{"role": "user", "content": f"{task_input}"}] | |||
| ).choices[0].message.content | |||
|
|
|||
| judgment.get_current_trace().async_evaluate( | |||
| judgment.async_evaluate( | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The example for Online Evaluations has been updated from judgment.get_current_trace().async_evaluate(...) to judgment.async_evaluate(...). This simplification in the API call is a positive change for usability!
Could you please confirm that judgment.async_evaluate is the current recommended public API and that it correctly handles the trace context as this change implies? Ensuring the accuracy of code examples is crucial for developers using the library.
README.md
Outdated
| @@ -170,18 +178,42 @@ Run Judgment on your own infrastructure: we provide comprehensive self-hosting c | |||
| 2. Use the [Judgment CLI](https://github.com/JudgmentLabs/judgment-cli) to deploy your self-hosted environment | |||
| 3. After your self-hosted instance is setup, make sure the `JUDGMENT_API_URL` environmental variable is set to your self-hosted backend endpoint | |||
|
|
|||
| ## 📚 Cookbooks | |||
|
|
|||
| Have your own? We're happy to feature it if you create a PR or message us on [Discord](https://discord.gg/taAufyhf). | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's great to encourage community contributions for Cookbooks and provide a Discord link! The current link is https://discord.gg/taAufyhf.
Could you verify if this is a permanent, non-expiring invite link for the community? Using a stable invite link would ensure long-term accessibility for users wishing to join the Discord server and contribute.
📝 Summary
Quick update to README: try making clustering image dark, change a few wordings