New – EC2 Instances (G5) With NVIDIA A10G Tensor Core GPUs

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/new-ec2-instances-g5-with-nvidia-a10g-tensor-core-gpus/

Two years ago I told you about the then-new G4 instances, which featured up to eight NVIDIA T4 Tensor Core GPUs. These instances were designed to give you cost-effective GPU power for machine learning inference and graphics-intensive applications.

Today I am happy to tell you about the new G5 instances, which feature up to eight NVIDIA A10G Tensor Core GPUs. Powered by second generation AMD EPYC processors, these instances deliver up to 40% better price-performance for inferencing and graphics-intensive operations in comparison to their predecessors.

On the GPU side, the A10G GPUs deliver to to 3.3x better ML training performance, up to 3x better ML inferencing performance, and up to 3x better graphics performance, in comparison to the T4 GPUs in the G4dn instances. Each A10G GPU has 24 GB of memory, 80 RT (ray tracing) cores, 320 third-generation NVIDIA Tensor Cores, and can deliver up to 250 TOPS (Tera Operations Per Second) of compute power for your AI workloads.

TrendingTäht: The state of fq_codel and sch_cake worldwide

Here are the specs:

Instance Name	NVIDIA A10G Tensor Core GPUs	vCPUs	Memory	Local Storage	EBS Bandwidth	Network Bandwidth
g5.xlarge	1	4	16 GiB	250 GB	Up to 3.5 Gbps	Up to 10 Gbps
g5.2xlarge	1	8	32 GiB	450 GB	Up to 3.5 Gbps	Up to 10 Gbps
g5.4xlarge	1	16	64 GiB	600 GB	8 Gbps	Up to 25 Gbps
g5.8xlarge	1	32	128 GiB	1900 GB	16 Gbps	25 Gbps
g5.12xlarge	4	48	192 GiB	3800 GB	16 Gbps	40 Gbps
g5.16xlarge	1	64	256 GiB	1900 GB	16 Gbps	25 Gbps
g5.24xlarge	4	96	384 GiB	3800 GB	19 Gbps	50 Gbps
g5.48xlarge	8	192	768 GiB	7600 GB	19 Gbps	100 Gbps

Like their predecessors, these instances are a great fit for many interesting types of workloads. Here are a few examples:

Media and Entertainment – Customers can use G5 instances to support finishing and color grading tasks, generally with the aid of high-end pro-grade tools. These tasks can also support real-time playback, aided by the plentiful amount of EBS bandwidth allocated to each instance. Customers can also use the increased ray-tracing power of G5 instances to support game development tools.

Remote Workstations – Customers in many different industries including Media and Entertainment, Gaming, Education, Architecture, Engineering and Construction want to run high-end graphical workstations in the cloud, and are looking for instances that come in a broad array of sizes.

Machine & Deep Learning – G5 instances deliver high performance and significant value for training and inferencing workloads. They also offer access to NVIDIA CuDNN, NVIDIA TensorRT, NVIDIA Triton Inference Server, and other ML/DL software from the NVIDIA NGC catalog, which have all been optimized for use with NVIDIA GPUs.

Autonomous Vehicles – Several of our customers are designing and simulating autonomous vehicles that include multiple real-time sensors. The customers make use of ray tracing to simulate sensor input in real time, and also gather data from real-world tests using tools that benefit from powerful networking and large amounts of memory.

The instances support Linux and Windows, and are compatible with a very long list of graphical and machine learning libraries including CUDA, CuDNN, CuBLAS, NVENC, TensorRT, OpenCL, DirectX, Vulkan, and OpenGL.

Available Now The new G5 instances are available now and you can start using them today in the US East (N. Virginia), US West (Oregon), and Europe (Ireland) Regions in On-Demand, Spot, Savings Plan, and Reserved Instance form. You can also launch them in Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (EKS) clusters,

To learn more, check out the G5 Instances page.

— Jeff;