Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump

Excited to announce Swiftide 0.12 🚀 A Rust library for building AI applications using retrieval augmented generation.

Retrieving the most relevant information for a given query is the key challenge in when building AI applications. Research and our ow…


This content originally appeared on DEV Community and was authored by Timon Vonk

Excited to announce Swiftide 0.12 🚀 A Rust library for building AI applications using retrieval augmented generation.

Retrieving the most relevant information for a given query is the key challenge in when building AI applications. Research and our own experience shows that similarity search on vectors is not enough. The idea behind hybrid search is fairly simple:

  1. Retrieve n documents with similarity search

  2. Retrieve n documents with another kind of search (ie full text, sparse vectors)

  3. Rerank the documents for relevancy

  4. Take the top k documents

Swiftide can now do hybrid search with #Qdrant, supports filters in search, load parquet files for indexing (i.e. from #HuggingFace) and is a lot faster!

Trumpets and a big thanks to @ephraimkunz for his first contribution! Read the full release post here, check us out on github


This content originally appeared on DEV Community and was authored by Timon Vonk


Print Share Comment Cite Upload Translate Updates
APA

Timon Vonk | Sciencx (2024-09-13T15:59:55+00:00) Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump. Retrieved from https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/

MLA
" » Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump." Timon Vonk | Sciencx - Friday September 13, 2024, https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/
HARVARD
Timon Vonk | Sciencx Friday September 13, 2024 » Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump., viewed ,<https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/>
VANCOUVER
Timon Vonk | Sciencx - » Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/
CHICAGO
" » Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump." Timon Vonk | Sciencx - Accessed . https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/
IEEE
" » Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump." Timon Vonk | Sciencx [Online]. Available: https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/. [Accessed: ]
rf:citation
» Swiftide 0.12 – Hybrid Search, search filters, parquet loader, and a giant speed bump | Timon Vonk | Sciencx | https://www.scien.cx/2024/09/13/swiftide-0-12-hybrid-search-search-filters-parquet-loader-and-a-giant-speed-bump/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts