AWS Introduces P-EAGLE for Parallel Speculative Decoding on SageMaker AI
AWS unveiled P-EAGLE, a new method for parallel speculative decoding, which boosts throughput by up to 1.69x on real-world benchmarks. The technique is now available in Amazon SageMaker JumpStart.