Ideason building a better future
DavidTsaiIdeas
  1. Intro
  2. 01How the FDA got here
  3. 02Elsa, in use
  4. 03A warning letter
  5. 04Inspection questions
  6. Closing

What does the FDA’s own AI use mean for drug developers?

The FDA is using AI. What that means for regulated work

How the FDA uses AI on its own work, and the questions it will ask about yours.

David Tsai

By David Tsai

I lead clinical operations in biotech and write about how AI is changing clinical trials and drug development.

6 min read

Introduction

The FDA has been using AI inside the agency for more than a year, and it has set rules around that use as it goes. For anyone in drug development, the lesson is not to copy the FDA’s tools. It is to build AI workflows that can answer the questions the FDA will ask during an inspection.

01

How the FDA got here

Rules for industry’s use of AI, from 2019 on.

The FDA’s work on AI started with how industry uses it. In 2019 it published a discussion paper on regulating AI and machine learning in software used as a medical device, followed by an action plan in January 2021. In March and May 2023 the drug center published discussion papers on AI in drug manufacturing and drug development, and in 2024 it set up a CDER AI Council.

In January 2025 it issued draft guidance on using AI to support regulatory decisions for drugs and biologics. The core idea: define the context of use, then show the model is credible for it.

In 2025 the agency also started using AI on its own work, in the open.

Timeline of FDA rules for industry’s use of AI: a 2019 discussion paper, a 2021 action plan, 2023 discussion papers, 2025 draft guidance and 2026 guiding principlesThe same timeline with the FDA’s own use added: the 2025 pilot, Elsa, agentic AI, Elsa 4.0 and the new SI wording in November 2026The same timeline with the April 2026 warning letter that cites overreliance on AI
1 / 3
Figure 1Rules for industry, the FDA’s own use, and the first warning letter that cites AI.
02

Elsa, in use

The FDA’s own AI tools, and the rules around them.

The FDA has also set rules for its own use:

That is secure access, defined use, governance, human accountability and documentation, all in place around the tool.

03

A warning letter for AI use

From the pharma side: what went wrong at one manufacturer.

In April 2026 the FDA sent a warning letter to a drug manufacturer that had used AI agents to help it comply with FDA regulations. The agents wrote the firm’s drug specifications, procedures, and master production and control records.

Investigators found the firm had shipped product without process validation, which 21 CFR 211.100 requires. The firm said it didn’t know about the requirement because its AI agent never mentioned it. The FDA cited “overreliance on artificial intelligence.” It said that if the firm uses AI for this work again, any output from an AI agent must be reviewed and cleared by an authorized person in its quality unit.

The FDA didn’t object to the firm using AI. It objected that no one in the quality unit reviewed what the AI produced.

04

Questions to expect in an inspection

Seven questions a quality system should answer.

I admire how the FDA uses Elsa. It treats AI as something to use properly: to cut tedious work, help reviewers handle more information and leave more room for scientific judgment. But the quality system around the tool is what matters most.

At inspection time, the question won’t be whether a company used AI. It will be whether the company can explain exactly how the workflow worked. The FDA hasn’t published a full inspection framework for AI. Its existing quality expectations make the likely questions clear:

  1. What was it allowed to do? A general-purpose chatbot is not a use case. Define the task, the users, the inputs and outputs, and what it must not do.
  2. What happens if this output is wrong? Drafting help needs lighter controls than safety signal detection or a submission analysis.
  3. What data did it see? Where it came from, whether it was authorized for this use, which version, and whether it was fit for purpose. A system can be secure and still give unreliable output from poorly understood data.
  4. How was the whole workflow validated? Not just the model: the data pipeline, the version, the prompts where they matter, the interfaces, the human review and the failure modes.
  5. Can you reconstruct what happened? The input, the model and settings, the output, the reviewer’s assessment, any overrides and the final decision.
  6. Who reviewed it, and against what standard? “Human in the loop” can’t mean someone clicks approve at the end. The reviewer needs training, the underlying evidence, the authority to reject and a record of the review.
  7. When do you stop trusting it? Vendors change models and data drifts. Define change control, monitoring, retesting and the criteria to stop using it.

A note on pharmacovigilance. The FDA uses Elsa to summarize adverse events, which supports evaluating AI-assisted safety workflows. It doesn’t show that any system can screen literature end to end, find every reportable case or replace a qualified safety judgment. The FDA’s own use doesn’t validate a vendor, a model or a sponsor’s process.

In closing

The FDA’s use of AI is not blanket permission for companies to automate regulated work. It shows that AI-assisted work is now part of the operating environment, and that quality systems need to catch up.

The question for industry is moving from whether you use AI to whether you can explain exactly how the workflow works when someone asks.

Quality infrastructure is going to be everything.

Read nextWhat makes a clinical trial resilient?Building resilient clinical trials

Start a conversation.

About the author

David Tsai

I lead clinical operations in biotech and write about how AI is changing clinical trials and drug development.

I’ve spent the last 15 years in clinical research and drug development, with the first several years working at a large academic hospital enrolling patients onto clinical trials.

Views and opinions expressed here are my own and do not necessarily reflect the position of current or past employers.

Full background on davidts.ai ↗Opens in a new tab, so this page stays open.

Before you go

The animated version

This is the ideal way to view this piece. The animations are there to drive the points home.

If you get motion sick, scroll slowly.

Continue to the animated version