Amazon CEO Andy Jassy said AWS is underway building "primitive services," or discrete building blocks, for generative AI and that approach will ensure customers bring more workloads to the cloud service.
Jassy’s shareholder letter landed as Amazon appointed Andrew Ng to its board of directors. Ng is managing general partner of AI Fund. He was also the founder of DeepLearning.AI, co-founder of Coursera and an adjunct professor at Stanford. Ng also has worked with Baidu and Google Brain.
In his 2023 shareholder letter, Jassy spend a good amount of space talking about generative AI and AWS services. Jassy walks through how primitive services were in Amazon's 2003 Vision document and how AWS' approach emerged from a partnership with Target in the early 2000s where Amazon was the back end to Target's web site.
"Pursuing primitives is not a guarantee of success. There are many you could build, and even more ways to combine them. But a good compass is to pick real customer problems you’re trying to solve," said Jassy, who noted that this approach to primitives guides everything from logistics to supply chain to stores to Prime delivery to AWS.
Jassy said AWS is designing a set of primitives focused on the layers of generative AI. The bottom layer is compute with Nvidia and Amazon's in-house processors. SageMaker, which is for customers building their own foundational models, is another service that's driving AI workloads. He noted Workday has cut inference latency by 80% with SageMaker.
The middle layer is where Bedrock will come in. Jassy said:
"What customers have learned at this early stage of GenAI is that there’s meaningful iteration required to build a production GenAI application with the requisite enterprise quality at the cost and latency needed. Customers don’t want only one model. They want access to various models and model sizes for different types of applications. Customers want a service that makes this experimenting and iterating simple, and this is what Bedrock does, which is why customers are so excited about it."
- AWS' Matt Wood on model choice, orchestration, Q and evaluating LLMs
- AWS ups its investment in Anthropic as giants form spheres of LLM influence
- AWS Q4 revenue growth 13% as Amazon's results shine
- Amazon Generative AI Stack Offering Overview
- Amazon's Vogels says 'cost awareness is a lost art' as AWS launches optimization tools
- These overlooked AWS re:Invent launches could solve pain points
- AWS launches Amazon Q, makes its case to be your generative AI stack
Regarding the application layer, Jassy also outlined AWS approach. He cited services such as Amazon Q, Rufus, Alexa and other applications, but noted most applications will be built by third parties. AWS' spin on the application layer is worth noting. Jassy said:
"While we’re building a substantial number of GenAI applications ourselves, the vast majority will ultimately be built by other companies. However, what we’re building in AWS is not just a compelling app or foundation model. These AWS services, at all three layers of the stack, comprise a set of primitives that democratize this next seminal phase of AI, and will empower internal and external builders to transform virtually every customer experience that we know (and invent altogether new ones as well). We’re optimistic that much of this world-changing AI will be built on top of AWS."
Jassy also noted that AWS' move to help customers save money will pay off in the long run and deals are accelerating along with renewals and migrations.
Other takeaways from the Amazon shareholder letter:
Processes matter as Amazon has discovered in its robotics efforts in its fulfillment network. Jassy said:
"There are dozens of processes we seek to automate to improve safety, productivity, and cost. Some of the biggest opportunities require invention in domains such as storage automation, manipulation, sortation, mobility of large cages across long distances, and automatic identification of items. Many teams would skip right to the complex solution, baking in “just enough” of these disciplines to make a concerted solution work, but which doesn’t solve much more, can’t easily be evolved as new requirements emerge, and that can’t be reused for other initiatives needing many of the same components. However, when you think in primitives, like our Robotics team does, you prioritize the building blocks, picking important initiatives that can benefit from each of these primitives, but which build the tool chest to compose more freely (and quickly) for future and complex needs."
Amazon has built primitive services for everything from storage, trailer loading, pallet mobility and sortation along with AI models to optimize those parts.
Lowering the cost to serve. Jassy said Amazon has plenty of room to continue to lower costs for consumers and its margins. "We’ve challenged every closely held belief in our fulfillment network, and reevaluated every part of it, and found several areas where we believe we can lower costs even further while also delivering faster for customers," said Jassy. "Our inbound fulfillment architecture and resulting inventory placement are areas of focus in 2024, and we have optimism there’s more upside for us."