feat: implement pre-assessment node for DeepModeling workflow #2910
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
- Add PreAssessmentAgent to analyze user input relevance to database design - Implement three-way decision logic: irrelevant, insufficient, sufficient - Create preAssessmentNode with retry logic and timeline logging - Add routeAfterPreAssessment for conditional workflow branching - Update WorkflowState and LangGraph annotations for preAssessmentResult - Integrate preAssessment as new entry point before analyzeRequirements - Update workflow documentation and test expectations - All existing tests pass with new workflow structure Addresses GitHub issue #5278: improve handling of irrelevant input by adding pre-assessment before DeepModeling process. Co-Authored-By: hirotaka.miyagi@route06.co.jp <h.miyagi.cnw@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Join our Discord community for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
Updates to Preview Branch (devin/1754377586-pre-assessment-node) ↗︎
Tasks are run on every commit but only new migration files are pushed.
View logs for this Workflow Run ↗︎. |
|
📝 
|
Issue
Why is this change needed?
The current DeepModeling workflow processes any user input through the full schema generation pipeline, including irrelevant inputs like "Hello!" or incomplete requests. This leads to unnecessary processing and poor user experience. This PR implements a pre-assessment node that filters inputs before the full DeepModeling process, similar to how other AI tools like ChatGPT Deep Research and Claude Deep Research handle irrelevant requests.
What does this change do?
High-Level Changes
New PreAssessmentAgent: AI agent that classifies user input into three categories:
irrelevant: Not related to database design (e.g., greetings, unrelated topics)insufficient: DB-related but lacks detail to proceed (e.g., "I need a database")sufficient: DB-related with enough context to begin designUpdated Workflow: The DeepModeling process now starts with pre-assessment instead of requirements analysis:
Conditional Routing: Based on assessment results:
sufficient→ proceeds toanalyzeRequirements(normal flow)irrelevant/insufficient→ routes tofinalizeArtifacts(ends workflow)Technical Implementation
Core Components:
PreAssessmentAgent: Uses GPT-4o-mini with structured prompts for input classificationpreAssessmentNode: Workflow node with retry logic and timeline loggingrouteAfterPreAssessment: Conditional routing functionWorkflowStateand LangGraph annotations forpreAssessmentResultJSON Response Parsing:
Updated Workflow Diagram
graph TD;
start([start]):::first
preAssessment(preAssessment)
analyzeRequirements(analyzeRequirements)
saveRequirementToArtifact(saveRequirementToArtifact)
dbAgent(dbAgent)
generateUsecase(generateUsecase)
prepareDML(prepareDML)
validateSchema(validateSchema)
finalizeArtifacts(finalizeArtifacts)
end([end]):::last
start --> preAssessment;
dbAgent --> generateUsecase;
finalizeArtifacts --> end;
generateUsecase --> prepareDML;
prepareDML --> validateSchema;
saveRequirementToArtifact --> dbAgent;
preAssessment -.-> analyzeRequirements;
preAssessment -.-> finalizeArtifacts;
analyzeRequirements -.-> saveRequirementToArtifact;
analyzeRequirements -.-> finalizeArtifacts;
validateSchema -.-> dbAgent;
validateSchema -.-> finalizeArtifacts;
preAssessment -.-> preAssessment;
analyzeRequirements -.-> analyzeRequirements;
classDef default fill:#f2f0ff,line-height:1.2;
classDef first fill-opacity:0;
classDef last fill:#bfb6fc;
Files Changed
New Files
preAssessmentNode.ts- Main workflow node implementationrouteAfterPreAssessment.ts- Conditional routing logicpreAssessmentAgent/- Complete agent implementation with prompts, JSON parser, and core logicModified Files
createGraph.ts- Updated workflow entry point and routingtypes.ts&langGraphUtils.ts- Added preAssessmentResult to stateREADME.md&createGraph.test.ts- Updated documentation and testsHuman Review Checklist
Critical Areas to Review:
Prompt Engineering Quality (
preAssessmentAgent/prompts.ts):Workflow Routing Logic (
routeAfterPreAssessment.ts):JSON Parsing Robustness (
preAssessmentAgent/jsonParser.ts):Integration Points (
createGraph.ts):Error Handling (
preAssessmentNode.ts):Testing Considerations:
Potential Risks:
Link to Devin run: https://app.devin.ai/sessions/b7f232be53454fd6a245403f77fc23ef
Requested by: @MH4GF