How to use JSON mode with OpenAI

[khiconit]

Open AI and Ollama now support output format. Please support it soon

Documentation: https://platform.openai.com/docs/guides/structured-outputs?context=without_parse#json-mode

[marcusschiesser]

JSON mode is already supported via:

const llm = new OpenAI({
    model: "gpt-4o",
    additionalChatOptions: { response_format: { type: "json_object" } },
  });

but we should generally support structured output

[AVtheking]

but we should generally support structured output

Do you mean having a parameter for the structured output with zod schema ?

[marcusschiesser]

yes, and generally support non openai LLMs

[AVtheking]

but what about those llm which are not having support for structured output ? do you mean to add it as a optional field with a flag just like we have for tool call for the llm ?

[AVtheking]

Created a PR for this #1749 , let me know if the approach is correct

[marcusschiesser]

just saw we have an issue for structured output already: #1249 - converting this to a discussion