Skip to content

feat: implement pre-assessment node for DeepModeling workflow #2910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions frontend/internal-packages/agent/src/chat/workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ A **LangGraph implementation** for processing chat messages in the LIAM applicat
%%{init: {'flowchart': {'curve': 'linear'}}}%%
graph TD;
__start__([<p>__start__</p>]):::first
preAssessment(preAssessment)
analyzeRequirements(analyzeRequirements)
saveRequirementToArtifact(saveRequirementToArtifact)
dbAgent(dbAgent)
Expand All @@ -16,16 +17,19 @@ graph TD;
validateSchema(validateSchema)
finalizeArtifacts(finalizeArtifacts)
__end__([<p>__end__</p>]):::last
__start__ --> analyzeRequirements;
__start__ --> preAssessment;
dbAgent --> generateUsecase;
finalizeArtifacts --> __end__;
generateUsecase --> prepareDML;
prepareDML --> validateSchema;
saveRequirementToArtifact --> dbAgent;
preAssessment -.-> analyzeRequirements;
preAssessment -.-> finalizeArtifacts;
analyzeRequirements -.-> saveRequirementToArtifact;
analyzeRequirements -.-> finalizeArtifacts;
validateSchema -.-> dbAgent;
validateSchema -.-> finalizeArtifacts;
preAssessment -.-> preAssessment;
analyzeRequirements -.-> analyzeRequirements;
classDef default fill:#f2f0ff,line-height:1.2;
classDef first fill-opacity:0;
Expand Down Expand Up @@ -77,13 +81,14 @@ interface WorkflowState {

## Nodes

1. **analyzeRequirements**: Organizes and clarifies requirements from user input (performed by pmAnalysisAgent)
2. **saveRequirementToArtifact**: Processes analyzed requirements, saves artifacts to database, and syncs timeline (performed by pmAgent)
3. **dbAgent**: DB Agent subgraph that handles database schema design - contains designSchema and invokeSchemaDesignTool nodes (performed by dbAgent)
4. **generateUsecase**: Creates use cases for testing with automatic timeline sync (performed by qaAgent)
5. **prepareDML**: Generates DML statements for testing (performed by qaAgent)
6. **validateSchema**: Executes DML and validates schema (performed by qaAgent)
7. **finalizeArtifacts**: Generates and saves comprehensive artifacts to database, handles error timeline items (performed by dbAgentArtifactGen)
1. **preAssessment**: Analyzes user input to determine if it's related to database design and has sufficient information (performed by preAssessmentAgent)
2. **analyzeRequirements**: Organizes and clarifies requirements from user input (performed by pmAnalysisAgent)
3. **saveRequirementToArtifact**: Processes analyzed requirements, saves artifacts to database, and syncs timeline (performed by pmAgent)
4. **dbAgent**: DB Agent subgraph that handles database schema design - contains designSchema and invokeSchemaDesignTool nodes (performed by dbAgent)
5. **generateUsecase**: Creates use cases for testing with automatic timeline sync (performed by qaAgent)
6. **prepareDML**: Generates DML statements for testing (performed by qaAgent)
7. **validateSchema**: Executes DML and validates schema (performed by qaAgent)
8. **finalizeArtifacts**: Generates and saves comprehensive artifacts to database, handles error timeline items (performed by dbAgentArtifactGen)

## DB Agent Subgraph

Expand Down Expand Up @@ -147,6 +152,7 @@ graph.addNode('dbAgent', dbAgentSubgraph) // No retry policy - handled internall

### Conditional Edge Logic

- **preAssessment**: Routes to `analyzeRequirements` when input is sufficient for database design, routes to `finalizeArtifacts` for irrelevant or insufficient inputs, retries `preAssessment` with retry count tracking (max 3 attempts)
- **analyzeRequirements**: Routes to `saveRequirementToArtifact` when requirements are successfully analyzed, retries `analyzeRequirements` with retry count tracking (max 3 attempts), fallback to `finalizeArtifacts` when max retries exceeded
- **saveRequirementToArtifact**: Always routes to `dbAgent` after processing artifacts (workflow termination node pattern)
- **dbAgent**: DB Agent subgraph handles internal routing between designSchema and invokeSchemaDesignTool nodes, routes to `generateUsecase` on completion
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ export { saveRequirementToArtifactNode } from '../../../pm-agent/nodes/saveRequi
export { analyzeRequirementsNode } from './analyzeRequirementsNode'
export { finalizeArtifactsNode } from './finalizeArtifactsNode'
export { generateUsecaseNode } from './generateUsecaseNode'
export { preAssessmentNode } from './preAssessmentNode'
export { prepareDmlNode } from './prepareDmlNode'
export { validateSchemaNode } from './validateSchemaNode'
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
import type { RunnableConfig } from '@langchain/core/runnables'
import type { Database } from '@liam-hq/db'
import { PreAssessmentAgent } from '../../../langchain/agents/preAssessmentAgent'
import { WorkflowTerminationError } from '../../../shared/errorHandling'
import { getConfigurable } from '../shared/getConfigurable'
import type { WorkflowState } from '../types'
import { logAssistantMessage } from '../utils/timelineLogger'

export async function preAssessmentNode(
state: WorkflowState,
config: RunnableConfig,
): Promise<WorkflowState> {
const assistantRole: Database['public']['Enums']['assistant_role_enum'] = 'pm'
const configurableResult = getConfigurable(config)
if (configurableResult.isErr()) {
throw new WorkflowTerminationError(
configurableResult.error,
'preAssessmentNode',
)
}
const { repositories } = configurableResult.value

await logAssistantMessage(
state,
repositories,
'Analyzing your request to determine the best approach...',
assistantRole,
)

const preAssessmentAgent = new PreAssessmentAgent()

const assessmentResult = await preAssessmentAgent.generate(state.messages)

return assessmentResult.match(
async (assessmentData) => {
if (
assessmentData.reasoning?.summary &&
assessmentData.reasoning.summary.length > 0
) {
for (const summaryItem of assessmentData.reasoning.summary) {
await logAssistantMessage(
state,
repositories,
summaryItem.text,
assistantRole,
)
}
}

await logAssistantMessage(
state,
repositories,
assessmentData.response.response,
assistantRole,
)

const preAssessmentResult = {
decision: assessmentData.response.decision,
reasoning: assessmentData.response.reasoning,
response: assessmentData.response.response,
}

return {
...state,
preAssessmentResult,
}
},
async (error) => {
const currentRetryCount = state.retryCount['preAssessment'] || 0
const newRetryCount = currentRetryCount + 1

await logAssistantMessage(
state,
repositories,
`Having trouble analyzing your request (attempt ${newRetryCount}): ${error.message}. Let me try a different approach...`,
assistantRole,
)

return {
...state,
retryCount: {
...state.retryCount,
preAssessment: newRetryCount,
},
preAssessmentResult: undefined,
}
},
)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import type { WorkflowState } from '../types'

export const routeAfterPreAssessment = (
state: WorkflowState,
): 'analyzeRequirements' | 'finalizeArtifacts' => {
const result = state.preAssessmentResult

if (result?.decision === 'sufficient') {
return 'analyzeRequirements'
}

return 'finalizeArtifacts'
}
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,14 @@ export const createAnnotations = () => {
return Annotation.Root({
...MessagesAnnotation.spec,
userInput: Annotation<string>,
preAssessmentResult: Annotation<
| {
decision: 'irrelevant' | 'insufficient' | 'sufficient'
reasoning: string
response: string
}
| undefined
>,
analyzedRequirements: Annotation<
| {
businessRequirement: string
Expand Down
7 changes: 7 additions & 0 deletions frontend/internal-packages/agent/src/chat/workflow/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ import type { Repositories } from '../../repositories'
export type WorkflowState = {
messages: BaseMessage[]
userInput: string
preAssessmentResult?:
| {
decision: 'irrelevant' | 'insufficient' | 'sufficient'
reasoning: string
response: string
}
| undefined
analyzedRequirements?:
| {
businessRequirement: string
Expand Down
6 changes: 5 additions & 1 deletion frontend/internal-packages/agent/src/createGraph.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ describe('createGraph', () => {
const expectedMermaidDiagram = `%%{init: {'flowchart': {'curve': 'linear'}}}%%
graph TD;
__start__([<p>__start__</p>]):::first
preAssessment(preAssessment)
analyzeRequirements(analyzeRequirements)
saveRequirementToArtifact(saveRequirementToArtifact)
dbAgent(dbAgent)
Expand All @@ -15,16 +16,19 @@ graph TD;
validateSchema(validateSchema)
finalizeArtifacts(finalizeArtifacts)
__end__([<p>__end__</p>]):::last
__start__ --> analyzeRequirements;
__start__ --> preAssessment;
dbAgent --> generateUsecase;
finalizeArtifacts --> __end__;
generateUsecase --> prepareDML;
prepareDML --> validateSchema;
saveRequirementToArtifact --> dbAgent;
preAssessment -.-> analyzeRequirements;
preAssessment -.-> finalizeArtifacts;
analyzeRequirements -.-> saveRequirementToArtifact;
analyzeRequirements -.-> finalizeArtifacts;
validateSchema -.-> dbAgent;
validateSchema -.-> finalizeArtifacts;
preAssessment -.-> preAssessment;
analyzeRequirements -.-> analyzeRequirements;
classDef default fill:#f2f0ff,line-height:1.2;
classDef first fill-opacity:0;
Expand Down
33 changes: 32 additions & 1 deletion frontend/internal-packages/agent/src/createGraph.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@ import {
analyzeRequirementsNode,
finalizeArtifactsNode,
generateUsecaseNode,
preAssessmentNode,
prepareDmlNode,
saveRequirementToArtifactNode,
validateSchemaNode,
} from './chat/workflow/nodes'
import { routeAfterPreAssessment } from './chat/workflow/routing/routeAfterPreAssessment'
import { createAnnotations } from './chat/workflow/shared/langGraphUtils'
import { createDbAgentGraph } from './db-agent/createDbAgentGraph'
import { RETRY_POLICY } from './shared/errorHandling'
Expand All @@ -22,6 +24,9 @@ export const createGraph = () => {
const dbAgentSubgraph = createDbAgentGraph()

graph
.addNode('preAssessment', preAssessmentNode, {
retryPolicy: RETRY_POLICY,
})
.addNode('analyzeRequirements', analyzeRequirementsNode, {
retryPolicy: RETRY_POLICY,
})
Expand All @@ -42,13 +47,39 @@ export const createGraph = () => {
retryPolicy: RETRY_POLICY,
})

.addEdge(START, 'analyzeRequirements')
.addEdge(START, 'preAssessment')
.addEdge('saveRequirementToArtifact', 'dbAgent')
.addEdge('dbAgent', 'generateUsecase')
.addEdge('generateUsecase', 'prepareDML')
.addEdge('prepareDML', 'validateSchema')
.addEdge('finalizeArtifacts', END)

// Conditional edges for pre-assessment
.addConditionalEdges(
'preAssessment',
(state) => {
const MAX_RETRIES = 3
const retryCount = state.retryCount['preAssessment'] || 0

if (state.preAssessmentResult !== undefined) {
return routeAfterPreAssessment(state)
}

// If max retries exceeded → fallback to finalizeArtifacts
if (retryCount >= MAX_RETRIES) {
return 'finalizeArtifacts'
}

// Otherwise → retry preAssessment
return 'preAssessment'
},
{
preAssessment: 'preAssessment',
analyzeRequirements: 'analyzeRequirements',
finalizeArtifacts: 'finalizeArtifacts',
},
)

// Conditional edges for requirements analysis
.addConditionalEdges(
'analyzeRequirements',
Expand Down
Loading
Loading