The AI field was pretty dull and boring, and rarely made ripples far outside its own realm.
But since the rise of generative AI, pretty much all industries are affected. With OpenAI's ChatGPT kickstarting the trend, others followed suit. And among the many tech companies that experiment and develop generative AI, include Nvidia.
The chipmaker is trying to piggyback the generative AI hype, wishing to be one of the bigger players in the lucrative industry.
However, it's entering the market with its own unique solution.
Calling it the 'Chat with RTX', Nvidia is introducing it as an alternative to other online AI chatbots.
At first glance, the AI does pretty much the same thing, but that is just about it.
Create a personalized chatbot with the Chat with RTX tech demo. #AIonRTX
Quickly get tailored info from your own files and content by connecting your data to an LLM on RTX-Powered PCs for local, fast, generative AI.
Download demo here: https://t.co/jcWr2a3lCG pic.twitter.com/qgWbOlnoX5— NVIDIA Studio (@NVIDIAStudio) February 13, 2024
First of, Chat with RTX uses retrieval-augmented generation (RAG) and TensorRT-LLM.
The former means that users are able to give the Large Language Model information, in which it will use alongside its internal training to generate accurate responses to users' queries. And as for the latter, it builds using TensorRT engines that can utilize Nvidia's chips to more efficiently run AI applications.
The result is an LLM users can feed their own data to (.txt, .pdf, and .doc filetypes), and which users can then query on that data.
It can even utilize online resources, such as YouTube videos.
And second, Chat with RTX can run locally.
Unlike others, which run on the clouds and require internet connection to work, the LLM from Nvidia is designed to work offline.
Nvidia has made the AI downloadable for free.
This is because Chat with RTX is all about letting everyone tap into the power of Nvidia's own graphics cards for AI tasks.
Nvidia has made the AI downloadable for free.
Then comes the issues.
The AI is capable, and is indeed customizable. But in order to run locally, it requires a huge amount of resources.
According to Nvidia, it only requires a PC equipped with an RTX 30- or 40-series GPU, and at least 8GB of video random access memory, or VRAM.
And in order to run its AI locally, Nvidia warns that downloading all the necessary files will eat up a fair amount of storage — 50GB to 100GB, depending on the model(s) selected.
In other words, running Chat with RTX is not actually cheap.
The next thing, is something that inadvertently happens by design, in which the AI is only good to answer to questions users already know.
While the AI excels in summarizing large datasets, the product casts doubt everyone has such a need for that. Because it runs locally, average PC users may not fancy a 100GB app to tell them what they already know.
Regardless, Chat with RTX should appeal those who prefer to keep their content out of the cloud.