Packaging SemanticFinder for Integration

[konishon]

"One could package everything nicely and use it e.g. instead of JavaScript search engines such as Lunr.js (also being used in mkdocs-material)."

Hi @do-me,

I found this idea in your README really interesting. Do you have any immediate plans to package SemanticFinder as a library? I'm exploring ways to offload semantic search from the backend to the frontend and think this could be a great fit.

If it’s not on your roadmap, do you have any advice on how I could approach starting this?
Would like to hear your thoughts.

[do-me]

Hi @konishon,
if you would like to get involved or program something like this lets team up for a POC!
Could you narrow down your specifics a bit more?

1. Can the initial processing be done server-side or should this also be done client-side? If client-side, have a look here
2. Do you want to index always on the fly (like our Chrome extension does) or would you like to persist an index as I'm doing here: https://huggingface.co/datasets/do-me/SemanticFinder? I think this could already be the solution you're looking for. However, starting from scratch, I'd choose a different file format like parquet or something similar to lancedb. Maybe also Orama (Performance Benchmarks oramasearch/orama#523)

I can give you a few more pointers and share some idea I head for efficient frontend semantic search.

1. I am eagerly waiting for mkdocs-material's solution Towards better documentation search  squidfunk/mkdocs-material#6307 (comment). As to what I understand, I'd expect a very similar setup with one static index file, inferencing in the browser (or via built-in browser APIs) possibly with smaller embedding models like POTION from Minishlab.
2. A while ago, @lizozom packaged parts of SemanticFinder in an npm package, which might be a pointer. However, starting from scratch, there are some things I would improve (chunking, options for tranformers.js etc.). Can happily point you to all the components that worked best for me in the past 2-3 years.
3. I am currently researching ways to query huge remote static files (parquet and flatgeobuf) with httpfs for instant semantic search. DuckDB is impressive but not fast enough if it comes to huge files & vector search. LanceDB needs a particular kind of connection and does not yet work with static files only (but maybe for your use case that might be a great fit). I created an alternative POC here with first promising results but still plenty of things to improve: https://www.reddit.com/r/MachineLearning/comments/1gssov1/p_flatgeobuf_as_static_vector_database_using/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button