Amazon Designs Custom Chips for Generative AI

An unmarked office building in Austin, Texas houses Amazon employees who are working on the design of two types of microchips, Inferentia and Trainium, aimed at training and accelerating generative AI. These chips provide an alternative to training large language models on Nvidia GPUs, which have become increasingly difficult and expensive to obtain. According to Amazon Web Services CEO Adam Selipsky, the company is in a prime position to meet the growing demand for chips in the generative AI field.

Competition in the Generative AI Market

While Amazon is working to catch up in the generative AI market, other companies have acted more quickly and invested more heavily. Microsoft gained attention for hosting the viral chatbot ChatGPT and investing $13 billion in OpenAI. Google also launched its own large language model, Bard, and invested $300 million in OpenAI competitor Anthropic. However, Amazon believes its custom silicon chips could give it an edge in the long run.

The Role of Custom Silicon in Generative AI

Amazon’s custom silicon, including Trainium and Inferentia, could provide a significant advantage in generative AI. These chips offer technical capabilities that competitors like Microsoft do not have. Amazon has been producing custom silicon since 2013, starting with the Nitro hardware, which is now the highest-volume chip used in AWS servers. In addition to chips, Amazon has also launched its Arm-based server chip, Graviton, and AI-focused chips.

Trainium and Inferentia: Amazon’s Custom Chips

Trainium, released in 2021, provides a 50% improvement in price performance for training machine learning models on AWS compared to other methods. Inferentia, released in 2019, is now in its second generation and offers low-cost, high-throughput, low-latency machine learning inference. While Nvidia GPUs are still dominant in training models, Amazon’s custom chips show promise for the future.

Amazon’s Cloud Dominance and Developer Tools

Amazon’s cloud computing platform, AWS, holds 40% of the market share in 2022. This dominance, combined with a growing portfolio of developer tools focused on generative AI, sets Amazon apart from its competitors. AWS customers who are already familiar with the platform may be drawn to Amazon for their generative AI needs. Amazon’s tools include Bedrock, which allows access to large language models from various providers, as well as AWS HealthScribe, SageMaker, and CodeWhisperer.

The Future of Generative AI on AWS

While AWS currently has over 100,000 customers using machine learning, analysts believe that number could grow significantly. Amazon is accelerating its push into generative AI, providing solutions architects, engineers, strategists, and data scientists to work with customers one-on-one. Despite concerns about data security with public large language models, Amazon ensures that its approach to generative AI and the Bedrock service offer isolated and encrypted environments with strict access controls.