Update README.md
Browse files
README.md
CHANGED
|
@@ -10,10 +10,10 @@ language:
|
|
| 10 |
<p align="left">
|
| 11 |
<strong>Technical report (coming soon)</strong> ·
|
| 12 |
<a href="https://github.com/Tencent-BAC/FastMTP"><strong>Github</strong></a> ·
|
|
|
|
| 13 |
<a href="https://modelscope.cn/models/TencentBAC/FastMTP"><strong>ModelScope</strong></a>
|
| 14 |
</p>
|
| 15 |
|
| 16 |
-
|
| 17 |
## Overview
|
| 18 |
|
| 19 |
FastMTP is a simple yet effective method that enhances Multi-Token Prediction (MTP) for speculative decoding during inference. Our approach fine-tunes a single MTP head with shared weights across multiple causal draft steps, enabling it to capture longer-range dependencies and achieve higher acceptance rates in speculative decoding. By incorporating language-aware vocabulary compression, we further reduce computational overhead during draft generation. Experimental results across diverse benchmarks demonstrate that FastMTP achieves an average of 2.03× speedup over vanilla next token prediction while maintaining lossless output quality. With low training cost and seamless integration into existing inference frameworks, FastMTP offers a practical and rapidly deployable solution for accelerating LLM inference.
|
|
|
|
| 10 |
<p align="left">
|
| 11 |
<strong>Technical report (coming soon)</strong> ·
|
| 12 |
<a href="https://github.com/Tencent-BAC/FastMTP"><strong>Github</strong></a> ·
|
| 13 |
+
<a href="https://huggingface.co/TencentBAC/FastMTP"><strong>HuggingFace</strong></a> ·
|
| 14 |
<a href="https://modelscope.cn/models/TencentBAC/FastMTP"><strong>ModelScope</strong></a>
|
| 15 |
</p>
|
| 16 |
|
|
|
|
| 17 |
## Overview
|
| 18 |
|
| 19 |
FastMTP is a simple yet effective method that enhances Multi-Token Prediction (MTP) for speculative decoding during inference. Our approach fine-tunes a single MTP head with shared weights across multiple causal draft steps, enabling it to capture longer-range dependencies and achieve higher acceptance rates in speculative decoding. By incorporating language-aware vocabulary compression, we further reduce computational overhead during draft generation. Experimental results across diverse benchmarks demonstrate that FastMTP achieves an average of 2.03× speedup over vanilla next token prediction while maintaining lossless output quality. With low training cost and seamless integration into existing inference frameworks, FastMTP offers a practical and rapidly deployable solution for accelerating LLM inference.
|