Meta Platforms 2023 of efficiency concluded with blowout fourth quarter earnings, a dividend and strong usage across the company's platform. But Meta CEO Mark Zuckerberg's AI ambitions may be the thing to watch in the long run.
Now it's easy to overlook Zuckerberg's ode to AI and open source. Meta’s fourth quarter revenue was $40.1 billion, up 25% from a year ago, and net income was $14.02 billion, up 200% from a year ago. The outlook was strong, and Meta will pay out its first-ever dividend. I could even find something nice to say about Meta's Reality Labs unit in that it had revenue of $1 billion in the fourth quarter (still lost $4.65 billion in the quarter).
Nevertheless, Meta's AI ambitions are clear and on par with Microsoft, Google and Amazon. Meta said 2024 capital expenditures will be $30 billion to $37 billion with a big chunk of that driven by AI-related spending. Here's a look at the AI strategy.
Meta plans to give consumers a copilot. The copilot craze has mostly been an enterprise effort, led by Microsoft, but Zuckerberg's vision was broader. He said:
"We'll be building the most popular and most advanced AI products and services. And if we succeed, everyone who uses our services will have a world-class AI assistant to help get things done, every creator will have an AI that their community can engage with, every business will have an AI that their customers can interact with to buy goods and get support, and every developer will have a state-of-the-art open-source model to build with."
You could call Meta's strategy a copilot for all play.
AI-centric devices won't be today's devices. Naturally, Zuckerberg will talk up smart glasses and Oculus, but the idea is broader. "Everyone will want a new category of computing devices that let you frictionlessly interact with AIs that can see what you see and hear what you hear, like smart glasses. And one thing that became clear to me in the last year is that this next generation of services requires building full general intelligence," he said.
AI models will have to be broad. Zuckerberg pushed back on use case specific AI products because they only do a subset of activities. Life doesn't operate in silos. "We're going to need our models to be able to reason, plan, code, remember and many other cognitive abilities in order to provide the best versions of the services that we envision. We've been working on general intelligence research and FAIR for more than a decade. But now general intelligence will be the theme of our product work as well," he said.
This AI strategy will need hyperscale infrastructure and Meta has it. "By the end of this year, we'll have about 350,000 (Nvidia) H100s, and including other GPUs, that will be around 600,000 H100 equivalents of compute. We're well positioned now because of the lessons that we learned from Reels. We initially underbuilt our GPU clusters for Reels. And when we were going through that, I decided that we should build enough capacity to support both Reels and another Reels-sized AI service that we expected to emerge so we wouldn't be in that situation again," said Zuckerberg.
Infrastructure will be critical since training and operating models are going to be more compute-intensive than today. Zuckerberg added he wasn't sure how much compute will be needed but state-of-the-art large language models have been trained on 10x the compute each year.
Zuckerberg said Meta is looking at novel data center designs as well as its own custom silicon for its workloads.
Long-term AI R&D. "While we're working on today's products and models, we're also working on the research that we need to advance for Llama 5, 6 and 7 in the coming years and beyond to develop full general intelligence," said Zuckerberg. "It's important to have a portfolio of multiyear investments in research projects, but it's also important to have clear launch vehicles like future Llama models that help focus our work."
Why Llama 2, open-source models change the LLM game
Open source is a big part of Meta's long-term AI strategy. Meta open sources its Llama models and tools like PyTorch as well as hardware designs. "Open sourcing improves our models. And because there's still significant work to turn our models into products because there will be other open-source models available anyway, we find that there are mostly advantages to being the open-source leader," said Zuckerberg.
He added that strategic benefits of open-source AI approaches include:
- Security.
- Compute efficiency.
- Ongoing feedback and scrutiny.
- Standardized approaches to hardware and software.
- Developer popularity and the ability to hire talent.
Training data and feedback loops will reinforce models and general intelligence AI. Zuckerberg said:
"When people think about data, they typically think about the corpus that you might use to train a model upfront. And on Facebook and Instagram, there are hundreds of billions of publicly shared images and tens of billions of public videos, which we estimate is greater than the common crawl data set. And people share large numbers of public text posts and comments across our services as well.
But even more important in the upfront training corpus is the ability to establish the right feedback loops with hundreds of millions of people interacting with AI services across our products. And this feedback is a big part of how we've improved our AI systems so quickly with Reels and Ads, especially over the last couple of years when we had to re-architect it around new rules."
Add it up and Meta plans to infuse AI into its platform for new services. Zuckerberg said the company will even put multiple teams on the same project to test versions.
To end the AI strategy talk, Zuckerberg also noted the metaverse investment. Rest assured that the bet is that AI and the metaverse will converge at some point. "I think that people are going to want new categories of devices that seamlessly engage with AIs frequently throughout the AI without having to take out your phone and press a button and point it at what you want to see," said Zuckerberg. "I think that smart glasses are going to be a compelling form factor for this, and it's a good example of how our AI and metaverse visions are connected."
Also see: How Generative AI Has Supercharged the Future of Work | Generative AI articles | Why you need a Chief AI Officer | Software development becomes generative AI's flagship use case | Enterprises seeing savings, productivity gains from generative AI | Work in a generative AI world will need critical, creative thinking