Inference Frameworks
TensorRT-LLM
nvidiaNVIDIA's fused-kernel LLM inference library — in-flight batching, FP8/INT4 quantization, and speculative decoding, optimized for Hopper/Blackwell
Editorial compatibility: these components can be used together in representative deployments. That's not a requirement or a validated implementation. For real deployments, see the sourced reference architectures.