Accelerator Hardware

Inferentia2

aws

AWS's cost-efficient, inference-only accelerator — 32 GB HBM, 190 TFLOPS FP16

Editorial compatibility: these components can be used together in representative deployments. That's not a requirement or a validated implementation. For real deployments, see the sourced reference architectures.

Compatible from earlier layers

AWS Neuron SDK

Compatible with later layers

AWS Inf2 (Inferentia2)