Howdy👋🏾, users of many AI models like ChatGPT reported hallucinations when asking for real-time stats during the Super Bowl. The issue is that AI learns from past data, not present events. At OpenAI DevDay, Sam Altman apologized that the data feeding ChatGPT lagged by years, promising to keep the data cutoff more current going forward.

This lag by design limits the types of queries current AI models can accurately answer. I need real-time info on the weather or live sports scores. Chatbots built on models like ChatGPT are trained on past data so they can’t update you on recent events like a rescheduled volleyball game. As IBM researchers noted, many foundation models essentially take closed-book tests, where that book is slightly out of date.

To address this, models can use Retrieval-Augmented Generation (RAG). RAG realizes when a question requires fresher data and fetches it from APIs, databases, or other real-time sources to augment the model’s existing knowledge. It’s like an open-book test that consults Twitter or Google for a boost. For example, the model could call the weather channel API to answer a question about next week’s weather to provide the most accurate forecast.

Perplexity AI exemplifies this approach. It understands questions via Natural Language Processing (NLP) to extract key points for search. It then scans its own data, APIs, and alternate sources to find insightful, up-to-date information to improve its answers, avoiding hallucination. This combines the power of foundation models with access to living data sources. This approach makes its results very accurate, and it is now one of my favorite places to ask complex questions on the Internet.

I’ve experimented with basic examples of using RAG with OpenAI’s Assistants in my playground test environment. Assistants allow you to define functions – a form of RAG – that get called when the context of a question matches the description you provide.

In the code block above, I define a weather function and provide details to help the AI model know when to utilize it. The code specifies required or optional parameters like “the city” to get the weather. If I ask “Should I wear shorts or a jacket to New Orleans next weekend?” the Assistant can infer I want current weather data. It will invoke my function, passing “New Orleans” as the city parameter and potentially detecting that I need a future weather timeframe.

In my playground, Mavis “Ace” Jarvis, my personal assistant, will take requests for weather or stock prices and call open free APIs to retrieve the latest data. It then uses those real-time responses to augment its knowledge when formulating a final answer.

We could expand this concept across many data sources – like getting the rescheduled volleyball game time or the status of support tickets from systems like Jira and Zendesk. For example, on my playground, Samantha “Smiles” Miles calls a mock API that returns a list of open tickets. It allows you to then ask for updates on any of those tickets. This mock API demonstrates how we could tie into various real-time internal data systems – from prices to employee PTO data – to enable richer responses.

Foundation models are amazing, but the real power of AI comes from a robust multi-model strategy. This approach combines AI models with external data sources like databases, APIs, CRMs, and websites that provide fresh, specialized information. The AI models handle what they have been explicitly trained for, while the living data sources supplement and augment that knowledge. With this combined approach, you can deploy solutions that avoid hallucination and stay current, delivering accurate responses no matter the question. Now, onto my thoughts on tech & things:

⚡️App Clips (instant apps on Android) are drastically underused. This feature allows a QR code or map link to trigger a thin version of an app that does not require a user to download the full app. For example, imagine scanning a QR code on a bill and paying it with an app that’s instantly downloaded. Play has a pretty amazing use case for App Clips that allows anyone to prototype an app and distribute it to investors using this feature. Absolutely a brilliant idea!

⚡️Epic has decided to accept Apple’s offer and release its App Store in the EU, potentially making Fortnite available for installation again for iPhone users in the EU. Apple’s rules charge per install, but Epic’s model is less about installs and app updates and more on SaaS services and virtual currency. More to come, but it is a place to keep an eye on.

⚡️If you missed it, Mark Zuckerberg gave his review of Apple Vision Pro, and surprise he thanks Meta Quest 3 is a better bang for the buck. I shared my thoughts on my blog, and I think Meta needed competition in the mixed-reality space, and this renewed competition will improve the next versions.

OpenAI functions are one way to implement RAG. Cloud platforms like Azure, GCP, and AWS also offer options to connect AI models to live data sources easily. When architecting AI, view it as part of a broader stack, leveraging two tiers of data: relatively static information that trains models and real-time, augmenting data via RAG.

For example, you could train a product support model on a database of specs, descriptions, etc. But data like prices and inventory change frequently, so supplementing with current API data enables accurate responses about new or out-of-stock items. The key is strategically segmenting data as an evergreen foundation vs dynamic augmentation.

An effective strategy for ingesting and partitioning training vs real-time data is just as crucial as data quality. Don’t ask AI models closed-book questions when you can provide open-book advantages with the right architecture. Apply RAG principles to give your models an unfair, data-fueled edge!

-jason

p.s. CaliExpress, a restaurant in Pasadena, CA, has opened a fully autonomous AI – powered restaurant. Yep, you heard that correctly. Fries and burgers are cooked by robots and ordered through an interactive kiosk. I need to schedule a trip out west for some taste testing.