Retrieval-augmented generation was a breakthrough in how LLMs source data to form its responses. Originally, an LLM was limited to its training data and context window, leading to gaps in its knowledge and a high rate of hallucinations (attempts by the AI to come up with answers where they don't clearly exist). This new approach (coined "RAG") allows the LLM to combine its foundational data with outside sources, dramatically increasing the data connections it can draw from.
From a UX perspective, this was also a breakthrough in the user experience of AI, as users could now guide the data that the AI referenced. Whereas a model's training data is private and difficult to parse for specific references, RAG gives the user transparency into the other data the AI is using.
One of the first products to take advantage of this technology was Perplexity.ai, which broke through with its ability to provide contextual search results to users alongside an AI chatbot that the user could talk to in order to refine or expand its results.
This pattern soon caught on, supported by complementary patterns like citations and the wayfinding pattern of using an initial reference source to frame the first prompt iteration. As this pattern proliferated, it expanded to include new components, like the ability to connect multiple private sources for the AI to reference, while protecting their data from being incorporated into the LLM primary training data.
Giving users the ability to see and manage sources is standardizing as a pattern. Some tools offer pro accounts to add or expand the number of sources that can be references. ChatGPT, Notion, Github, and Microsoft are just some of the major players that allow users to connect the conversational LLM to their personal or proprietary data, helping to connect fragmented data across the enterprise and decreasing the amount of time users spend looking for answers and resources to common questions.
RAG use cases
The use cases for RAG seem almost infinite, from learning and development to sales intelligence to product management. Consider how integrating related patterns can help users even further:
- Filtering sources by account or keyword could let employees ask questions about a specific account, from insights in Productboard, to notes in Salesforce, to account based marketing in Hubspot and more, combined with public knowledge. Could a team prompt the LLM for a summary? Or perhaps ask it to generate a microsite that the account team can easily reference?
- Generating custom built courses for internal enablement around new features, referencing competitive and market research, product development and strategic docs, product marketing and technical writing, and supported by an internal framework on skills developed by learning professionals. Using this information, the AI could quiz employees on key details, record and grade practice demos for people getting ramped up, and help create personalized talk tracks for leaders on overall areas of weakness.
- Combining sources across customers with a customer experience framework to support a voice of the customer program that mimics real customers as personas through AI Characters. Designers might pressure test initial concepts, working rapidly and iteratively with the AI through the first few days of the sprint while preparing a prototype for research and evaluation with actual customers. Using Gong recordings and other insight sources, a team might work backwards to identify the customers most likely to be experiencing the problems they are addressing, allowing account teams to proactively engage those customers so they feel like co-creators of a solution instead of waiting until the pain becomes acute.
