Skip to content

Commit 8cbdfb4

Browse files
committed
misread
1 parent caf11b7 commit 8cbdfb4

File tree

1 file changed

+0
-30
lines changed

1 file changed

+0
-30
lines changed

README.md

Lines changed: 0 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -100,25 +100,6 @@ quantized, indices, commit_loss = residual_vq(x)
100100
# (1, 1024, 256), (2, 1, 1024, 8), (2, 1, 8)
101101
```
102102

103-
104-
<a href="https://arxiv.org/abs/2305.05065">This paper</a> out of Google Deepmind claims that residual vector quantization can induce hierarchical semantic ids for building a recommender system. In their scheme, they use increasing number of codes across depth for it to work. This repository supports that scheme as so
105-
106-
```python
107-
import torch
108-
from vector_quantize_pytorch import ResidualVQ
109-
110-
residual_vq = ResidualVQ(
111-
dim = 256,
112-
codebook_size = (5, 128, 256), # from top most hierarchy to lowest, 5 codes, 128 codes, then 256 codes
113-
)
114-
115-
x = torch.randn(2, 16, 256)
116-
117-
quantized, indices, commit_loss = residual_vq(x)
118-
119-
# (2, 16, 256), (2, 16, 3), (2, 3)
120-
```
121-
122103
## Initialization
123104

124105
The SoundStream paper proposes that the codebook should be initialized by the kmeans centroids of the first batch. You can easily turn on this feature with one flag `kmeans_init = True`, for either `VectorQuantize` or `ResidualVQ` class
@@ -733,14 +714,3 @@ assert loss.item() >= 0
733714
url = {https://api.semanticscholar.org/CorpusID:273229218}
734715
}
735716
```
736-
737-
```bibtex
738-
@article{Rajput2023RecommenderSW,
739-
title = {Recommender Systems with Generative Retrieval},
740-
author = {Shashank Rajput and Nikhil Mehta and Anima Singh and Raghunandan H. Keshavan and Trung Hieu Vu and Lukasz Heldt and Lichan Hong and Yi Tay and Vinh Q. Tran and Jonah Samost and Maciej Kula and Ed H. Chi and Maheswaran Sathiamoorthy},
741-
journal = {ArXiv},
742-
year = {2023},
743-
volume = {abs/2305.05065},
744-
url = {https://api.semanticscholar.org/CorpusID:258564854}
745-
}
746-
```

0 commit comments

Comments
 (0)