Skip to content

Commit ecc8c71

Browse files
authored
Merge pull request #202 from cfifty/master
Update Sim VQ reference in README.
2 parents ae5e07c + c67b566 commit ecc8c71

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ This repository should also automatically synchronizing the codebooks in a multi
280280

281281
<img src="./images/simvq.png" width="400px"></img>
282282

283-
A <a href="https://arxiv.org/abs/2411.02038">new ICLR 2025 paper</a> proposes a scheme where the codebook is frozen, and the codes are implicitly generated through a linear projection. The authors claim this setup leads to less codebook collapse as well as easier convergence. I have found this to perform even better when paired with <a href="https://arxiv.org/abs/2410.06424">rotation trick</a> from Fifty et al., and expanding the linear projection to a small one layer MLP. You can experiment with it as so
283+
A <a href="https://arxiv.org/abs/2411.02038">new paper</a> proposes a scheme where the codebook is frozen, and the codes are implicitly generated through a linear projection. The authors claim this setup leads to less codebook collapse as well as easier convergence. I have found this to perform even better when paired with <a href="https://arxiv.org/abs/2410.06424">rotation trick</a> from Fifty et al., and expanding the linear projection to a small one layer MLP. You can experiment with it as so
284284

285285
Update: hearing mixed results
286286

0 commit comments

Comments
 (0)