[WIP] Support DeepSeek-V3.1 tool call

[Xu-Wenqing]

Motivation

Support tool call for DeepSeek-V3.1

Modifications

1. Add tool parser "deepseekv31"
2. Add a new chat template

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[jinmingyi1998]

does it work?

[freedomkk-qfeng]

does it work?

not work, here's an example:

request

{
  "messages": [
    {
      "role": "user",
      "content": "Shanghai weather"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Useful when you want to query the weather of a specified city.",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city or district, e.g., Beijing, Hangzhou, Yuhang District."
            }
          }
        },
        "required": [
          "location"
        ]
      }
    }
  ],
  "stream": false,
  "model": "deepseek-r1"
}

response

{
  "id": "708ff5439c474fda8f73f2656005a18c",
  "object": "chat.completion",
  "created": 1755798766,
  "model": "deepseek-r1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'll check the weather in Shanghai for you.<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>get_current_weather<｜tool▁sep｜>{\"location\": \"Shanghai\"}<｜tool▁call▁end｜><｜tool▁calls▁end｜>",
        "reasoning_content": null,
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "tool_calls",
      "matched_stop": null
    }
  ],
  "usage": {
    "prompt_tokens": 176,
    "total_tokens": 203,
    "completion_tokens": 27,
    "prompt_tokens_details": null,
    "reasoning_tokens": 0
  },
  "metadata": {
    "weight_version": "default"
  }
}

[Xu-Wenqing]

@jinmingyi1998 @freedomkk-qfeng
The tool call format of DeepSeek-V3.1 is different from DeepSeek-V3/R1:
DeepSeek-V3.1: <｜tool▁calls▁begin｜><｜tool▁call▁begin｜>tool_call_name<｜tool▁sep｜>tool_call_arguments<｜tool▁call▁end｜><｜tool▁calls▁end｜>
DeepSeek-R1/V3: <｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>FUNCTION_NAME\n'```json\n{"param1": "value1", "param2": "value2"}\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜>
So we can't use --tool-call-parser deepseekv3 for this DeepSeek-V3.1.

I will write a new tool parser to support DeepSeek-V3.1 CC @CatherineSue @zhyncs @JustinTong0323