Vector DB implementation

[xav-db]

We are currently implementing the vector datatypes for helix. I have implemented a first version of the HNSW algorithm. It is pretty fast so far but there are likely numerous optimisations that can be made. This discussion is for ideas about how we can make the vector db side of helix even faster. @lukasnxyz

[lukasnxyz]

openai embedding models:

• small dim size: 1536
• large dim size: 3072
• dim reduction is also possible via the api so might have to implement that better with current version of dim reduction in vector_optims branch

https://platform.openai.com/docs/guides/embeddings?lang=python#embedding-models