Skip to content

Commit 76ec1de

Browse files
authored
update
1 parent 3e4ce16 commit 76ec1de

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,8 @@ This repository should also automatically synchronizing the codebooks in a multi
282282

283283
A <a href="https://arxiv.org/abs/2411.02038">new ICLR 2025 paper</a> proposes a scheme where the codebook is frozen, and the codes are implicitly generated through a linear projection. The authors claim this setup leads to less codebook collapse as well as easier convergence. I have found this to perform even better when paired with <a href="https://arxiv.org/abs/2410.06424">rotation trick</a> from Fifty et al., and expanding the linear projection to a small one layer MLP. You can experiment with it as so
284284

285+
Update: hearing mixed results
286+
285287
```python
286288
import torch
287289
from vector_quantize_pytorch import SimVQ

0 commit comments

Comments
 (0)