The NVIDIA H100 GPU: Supercharged Computing Explained

The NVIDIA H100 GPU: Supercharged Computing Explained

12 August 2023

Boosting Performance Like Never Before

The NVIDIA© H100 Tensor Core GPU has made a revolutionary leap in computing power. This technological marvel is designed to provide unmatched performance, scalability, and security, making it an essential asset for AI.

So what is a GPU?

The device you're reading this on likely has a CPU (central processing unit), which you'll probably know as a Processor. It will also have a GPU (graphics processing unit), commonly referred to as a Graphics Card. Let's talk car engines to explain the difference:

• The Engine (CPU): Just as the engine is the heart of the car, powering it and keeping everything running smoothly, the CPU is the central brain of a computer, handling most of the basic operations and tasks.
• The Turbocharger (GPU): A turbocharger in a car takes the exhaust gases and uses them to boost the engine's power, making the car run faster and more efficiently. Similarly, a GPU takes specific tasks (like data processing) off the CPU and handles them more efficiently, boosting the computer's performance. The more memory your GPU has, the faster your computer will run.

By using the power of GPU(s), you can speed up both the training and inference (execution) of your AI model, allowing it to process more complex tasks. This is why GPU-powered AI Large Language Models (LLMs), like LLaMA 2, can process data so much faster than their CPU counterparts.

What Does the NVIDIA H100 Mean for Your Business?

1. Unparalleled Speed: The H100 allows up to 256 H100 GPUs to work together. This means extraordinary fast processing and the ability to handle massive tasks like never before.
2. Dedicated Language Processing: With its Transformer Engine, the H100 can tackle complex language models containing trillions of parameters. If you're looking to train and AI on huge datasets, this is the tool for you.
3. Accelerated Conversational AI: The innovative technology within the H100 can boost large language models (LLMs) by an astonishing 30X over previous models, paving the way for advanced conversational AI applications.

Why Should You Care?

• Powerful Processing: From handling customer queries to analysing complex data, the H100's capabilities can significantly enhance your business operations.
• Easy Integration: It's designed to fit seamlessly into existing data centre environment, making it a hassle-free addition.
• Future-Ready: Investing in the H100 means equipping your business with cutting-edge technology that's ready to meet tomorrow's challenges.

When's it available?

The bad news is that the new Nvidia's chips are in short supply as tech companies jostle for GPU capacity to develop their own AI models: Most chips are booked until at least the end of 2023. But the good news is that NVIDIA are partnering up with an AI tools company called Hugging Face, which will be rolling out a product in the coming months. It will be available to Premium subscribers on Hugging Face. Contact us for more information.

Will it reduce costs?

Or, why can running your own AI model be expensive to start with? First, a model is trained using large (or even huge) amounts of data, a process that can take months and can need thousands of GPUs. Then the model is used in software that users interact with, using a process called inference. Like training, inference is computationally expensive, and it requires a lot of processing power every time the software runs, for instance when it writes text for you, compares data or produces an image. But unlike training, inference takes place nearly constantly, while training is only needed when the model needs updating (with new documents or data, for example).

Nvidia's CEO Jensen Huang: "You can take pretty much any large language model you want and put it in this and it will inference like crazy. The inference cost of large language models will drop significantly."

What this means is that soon, every small business will be able to afford to have their own AI model, trained just for your business, keeping data on finances, customers, orders, products, competitors, competitor products and more. And as you grow, adding millions of rows of data won't break the bank or take months to train.