Recently, we have been, rightly so, bombarded 📢📢📢with news about LLMs and how they continue to improve their capabilities to predict the next word in a text or a pixel in a photo and video. The latest improvements in 🤖📽️AI video creation show its capability of predicting not just the right pixels needed to create an object like a football or person but also the natural movement of an object, making AI-created videos much more realistic.
But let’s take a step back and look at how a RAG-based AI uses your organisation’s data to answer questions from your employees or customers. How is AI able to find the right or relevant data to answer questions?
Finding documents the 🦖Old Skool way
In the old days, oh no, sorry, I mean, we do it now by creating folders where we save office documents related to the same topic, making it easier for users to find the relevant data. We all know how that turns out when you have massive amounts of documents created by all types of different users. Finding a document on SharePoint or a fileserver isn’t an easy task.
This is somewhat Old Skool and partly replaced by generic search functionality, enabling users to type in what they are looking for and make relevant documents appear. Great for when you know precisely what you are looking for, not so great if you want to expand the topic. This means multiple searches are needed, and the users should have a perfect understanding of what they are looking for, which is often not the case when exploring new topics.
Vectorstores
🚫It’s clear that directly connecting your SharePoint or Fileserver to a LLM doesn’t work. We need a much more efficient way of finding relevant data, enabling the LLM to answer your questions. Enter the Vectorstore. Vectorstores act as the bridge between your organisation’s data and the LLM, but they do much more than just pass through the data.
A vital feature of Vectorstores and the reason they are used in AI systems is their ability to store semantic information. This means that instead of just indexing documents by keywords or metadata, Vectorstores capture the meaning and context of the data. They do this by converting pieces of information—whether text, images, or other data—into high-dimensional vectors. These vectors are numerical representations that encode the semantic essence of the information.
Imagine having a vast library where every book is summarised and stored in a way that captures its core themes and ideas. This is what Vectorstores do for your data. When an LLM needs to answer a question, it can query the Vectorstore to retrieve not just any document containing the keywords but the most semantically relevant information. This allows for a much more nuanced and accurate response.
Keyword vs context
Traditional search methods rely heavily on keyword matching, which can overlook contextually relevant information. In contrast, Vectorstores enable AI systems to understand and leverage the relationships between different pieces of data, much like how our brains connect related concepts. This makes the AI’s responses richer and more contextually appropriate, transforming how we interact with and utilise vast amounts of organisational data. In essence, vectorstores are the silent powerhouses that enable modern AI to be not just smarter but more intuitively helpful.
See the simplified representation of a Vectorstore below. If the user creates a query with Cats as the subject, the LLM is able to find data in the Vectorstore that is closely related to the subject of Cats, while it will discard vectors (data) that represent an Apple.
Incorporating Text, Images, and Videos
One of the powerful features of modern vectorstores is their ability to handle multimodal data. This means they can store and manage not just text, but also images and videos, all within the same system. Using a Unified Embedding Space for seamless modality data integration enables LLMs to retrieve data across different modalities.
With Semantic Search Across Modalities, a user creates a query that is converted into a vector that can be matched against vectors from all types of data. This means that a textual query could retrieve relevant text documents, images, and even video clips that are semantically related to the query. This can work in all directions, feeding an image into the AI and getting text, video, and images as a result.
Conclusion
❗When your organisation is planning on using a RAG-based AI solution, the question is not if you use a Vectorstore, but how you can effectively vectorise your organisation’s data.
📢Vectorstores are a vital part of your RAG system, enabling the LLMs to provide more precise, context-aware responses and significantly enhancing applications such as customer support, where accurate and relevant information is crucial for addressing user inquiries effectively.
Related blog: Preprocessing data for your RAG model – Braindenburg Unlocking Enterprise Data Value with RAG-Powered AI Assistants