Skip to content

Commit 3864ee2

Browse files
author
liuzichang01
committed
fix docs
1 parent 164b83a commit 3864ee2

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

docs/zh/features/speculative_decoding.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,13 +108,13 @@ python -m fastdeploy.entrypoints.openai.api_server \
108108
```
109109

110110
## 🧠 使用 Ngram 解码
111-
该算法通过 n-gram 窗口从 prompt 和已生成的 Token 中进行匹配生成草稿 Token,适合输入和输出有很大 overlap 的场景如代码编辑、文档查询等查看论文地址
111+
该算法通过 n-gram 窗口从 prompt 和已生成的 Token 中进行匹配生成草稿 Token,适合输入和输出有很大 overlap 的场景,如代码续写、文档查询等
112112
> 使用 4×H100;量化方式选择 WINT4
113113
> 配置文件:benchmarks/yaml/eb45t-32k-wint4-mtp-h100-tp4.yaml
114114
```
115115
python -m fastdeploy.entrypoints.openai.api_server \
116116
--model ${path_to_main_model} \
117117
--tensor-parallel-size 4 \
118118
--config ${path_to_FastDeploy}benchmarks/yaml/eb45t-32k-wint4-mtp-h100-tp4.yaml \
119-
--speculative-config '{"method": "mtp", "num_speculative_tokens": 1, "model": "${mtp_model_path}"}'
119+
--speculative-config '{"method": "ngram", "num_speculative_tokens": 1, "model": "${mtp_model_path}"}'
120120
```

0 commit comments

Comments
 (0)