More and more companies are running large language models (LLMs), which require access to GPUs. The most popular are by far from Nvidia, making them expensive and often in short supply. Renting a long-term instance from a cloud provider when you only need access to these expensive resources for a single job doesn't necessarily make sense.
To help solve that problem, AWS has launched Amazon Elastic Compute Cloud (EC2) capacity blocks for ML, allowing customers to purchase access to these GPUs for a defined period of time, usually to run some type of related work. with AI, such as training a machine learning model or running an experiment with an existing model.
“This is an innovative new way to schedule GPU instances where you can reserve the number of instances you need for a future date just for the amount of time you need,” wrote Channy Yun in a post on the AWS blog announcing the new feature.
The product provides customers access to NVIDIA H100 Tensor Core GPU instances in cluster sizes from one to 64 instances with 8 GPUs per instance. They can reserve time for up to 14 days in 1-day increments, up to 8 weeks in advance. When the time period ends, the instances will be automatically shut down.
The new product allows users to sign up for as many instances as they need over a defined period of time, like booking a hotel room for a certain number of days (as the company put it). From the client's perspective, you'll know exactly how long the job will take, how many GPUs it will use, and how much it will cost up front, giving you cost certainty.
For Amazon, they can put these in-demand resources to work in a quasi-auction environment, ensuring revenue (assuming customers come, of course). The price of access to these resources will be truly dynamic, varying depending on supply and demand, according to the company.
When users sign up for the service, the total cost of the time period and resources is displayed. Users can increase or decrease it, depending on their appetite for resources and their budget before agreeing to purchase.
