Eli Lilly: Lessons from running an AI factory

Eli Lilly's AI factory is seeing strong usage running experiments at scale and the company has been able to save on token expenditures with control over its models and data.

Speaking at CoreWeave's Fully Connected 2026 conference, Brian Lewis, head of machine learning engineering and advanced intelligence, walked through the AI stack that powers Lilly's research efforts. Lewis' talk touched on multiple key issues in AI right now including the sovereignty, governance, open models that can be tailored and creating new ways of working for the 12,000 people in pharmaceutical research and development at Lilly.

Lilly announced LillyPod in February and its plans to transform its research operations. The AI factory, which is powered by Nvidia, is the most powerful supercomputer in the pharmaceutical industry for now. LillyPod features Nvidia DGX SuperPOD B300 and including 1,016 Nvidia Blackwell GPUs.

The pharmaceutical giant said it is using its own data, including information about millions of molecules, to tune models. Lilly also has Lilly TuneLab, which is an AI drug discovery platform. In addition, Lilly's AI factory runs disconnected from the internet.

The approach

Lilly treats AI as part of its scientific infrastructure and it's not just a set of models. Smith said AI must support discovery and meet standards for reproducing results and traceability.

Here's a look at the high-level approach.

Build AI efforts around the business mission and your standards. Smith emphasized that Lilly is a medicine company and to find breakthroughs and new medicines digital experiments need the same rigor as physical ones. Teams should be able to reconstruct how an output was produced and connect it back to its inputs. "Take the scientific method and apply it in the digital state," said Smith.

Prioritize disciplined engineering over chasing AI advances. Smith said Lilly is looking to benefit from new advances without rebuilding the tech stack repeatedly. Lilly built one AI operating system for LillyPod that includes the following layers that connect data, models and tools into a single system.

• Experience, which includes AI coding agents, scientific desktops and custom tools so scientists can leverage the system.
• Models and governed access. The model layer includes experiment tracking and a model registry with CoreWeave's Weights & Biases unit. Smith said Lilly routes models based on workload and cost. "We still use the cloud to some extent, but having the intelligent routing can decide based on workload and GPU hour where to send a job," said Smith.
• Compute, which is the Nvidia B300 GPUs with on-premises inference. Smith repeatedly noted that token costs weren't an issue given Lilly is running on-prem. "We can operate autonomously and locally without a connection to the internet," said Smith.
• Data, which includes high-performance storage purpose-built for AI workloads and shared across research teams.

"LillyPod is or greatest new scientific instrument," said Smith. "And our scientists don't have to wait for cloud GPUs to become available. They don't have to pay street price."

AI should fit scientist workflows. Smith said it was critical to connect LillyPod to the familiar tools and lab systems already being used. "We're not creating new portals, new apps and systems, but really integrating with the tools that people use," said Smith.

For instance, tracking training and inference runs, inputs, outputs and evaluations is the equivalent of a lab notebook. This record provides the reproducibility that shows how evidence informed a conclusion.

The experimentation ethos. Smith said Lilly uses the AI factory as a way to conduct experiments at scale. Failed or unexpected experiments are valuable if results inform the next attempt. Simply put, Lilly's AI infrastructure gives the company more at bats.

AI sits within IT. Smith's unit is within Lilly's IT group. "We serve all of the business functions within the company, including research and development, manufacturing, and commercial, as well as global services or enterprise functions, and we essentially build the brains that go into then our AI solutions, be they agentic or other problems," said Smith.

Sovereignty matters. LillyPod operates in an air-gapped environment disconnected from the internet. This air-gapped approach enables Lilly to use decades of proprietary data. "One of the primary reasons we made this massive investment in this hardware and partnered with Nvidia to make it possible was because we saw an opportunity to use it for finding the next therapies," said Smith.

Lessons learned

Smith also outlined some of the lessons from operating LillyPod. Here's a look:

Institutional knowledge matters. Smith said Lilly's AI infrastructure efforts benefited from retaining people who knew the company's data center environment and local compute even though most workloads moved to the cloud. "We had the core people we needed with some external partners to do this successfully," said Smith.

Technology shifts aren't usually all or nothing and Lilly was able to keep people that could operate as conditions shifted. Remember that workloads were on premises, moved to the cloud and now AI costs are bringing hybrid and local compute back.

Change management works better in a controlled environment. Smith said Lilly's AI experimentation was good for learnings, but created expectations that didn't always translate to scientific work. Local compute provided a controlled place to test. "The investment in local compute gave us the ability to try and fail with a little bit more comfort," said Smith.

Control your destiny. Lilly built its own AI infrastructure to train its own models on Nvidia's Nemotron and control its costs. "We are getting additional value out of our investment by running our models internally. It's more about controlling our own business than worrying about tokens," said Smith. "With agentic coding models we've been able to offset a good portion of our cloud model costs."

Scientists shouldn't have to manage infrastructure and worry about models. Smith said Lilly's approach is to connect developer tools and repositories to LillyPod while the routing layer picks the models. "We don't want employees to worry about which model to pick. We'll just essentially say you're using the Lilly model," said Smith.

In other words, model abstraction can keep people focused on the real work and business outcomes.

Governance and visibility is maturing. Smith said governance and visibility in the industry needs to mature. Lilly uses infrastructure boundaries to control risk and it's deliberate about evaluating models, but Smith said model inventory, observability and compute management are works in progress.

Research is leading AI adoption, but the goal is to broaden usage for more workflows. Smith said LillyPod's current utilization was 90% research and 10% enterprise, but that'll will shift over time. Smith said Lilly is iterating based on the feedback from its power users in research. Manufacturing is a promising AI use case.

Smith said:

"Based on what we see in utilization, LillyPod is running red hot and a lot of data and research scientists are doing their experiments. We will not know until we know which ones of those actually improve the human condition. But the bet has been made."

More from CoreWeave Fully Connected 2026

• How Capital One evaluates, governs its AI agents
• Caterpillar's AI autonomy efforts accelerate, but domain knowledge drives returns
• CoreWeave launches CoreWeave Forge: Here's a look at the strategy