How to Run a Security Review of an AI Automation Vendor

Automation vendors get evaluated on capability and price. The security review usually happens later, often as a questionnaire sent by someone who was not in the capability conversation, and it tends to produce a stack of documents that does not tell you much about the actual risk.

The risk in an AI automation deployment is rarely where the questionnaire looks. It is not usually the model provider, which is typically the most scrutinised and best documented part of the stack. It is the credentials the workflow holds, the intermediate data it writes, and who at the vendor can see any of it.

Here is a review that gets at those things, organised as the questions worth asking and the answers worth accepting.

Start by drawing the data path

Before any questionnaire, ask the vendor to draw the flow. Every hop, in order, from the moment data leaves your environment to the moment a result is written back.

A serious vendor can do this from memory on a call. The diagram should name every system the data touches, including infrastructure providers, model providers, and any queue, log, or storage that holds it in between. If the answer is vague at any hop, that hop is where you should focus the rest of the review.

Specific things the diagram must show:

• Which systems the workflow reads from, and at what scope. Read access to a mailbox and read access to a specific folder are very different exposures.
• Where data is processed. Whose infrastructure, in what region.
• Every place data comes to rest, even briefly. Queues, caches, temporary object storage, and error logs are all places data lives, and all get forgotten in security reviews.
• Which systems the workflow writes to, and with what permission.

The most common gap this exercise surfaces is logging. Workflows that handle sensitive documents often write the full document content into application logs for debugging. Those logs then sit in a third-party observability platform with a much broader access list than the workflow itself.

Separate the vendor's posture from the model provider's

These are two different reviews and conflating them is the most common mistake.

The model provider governs what happens to a prompt after it arrives. Major providers offer zero data retention and no training on business API traffic. This part is well documented, contractually addressable, and rarely where things go wrong.

The vendor governs everything else: what data goes into that prompt in the first place, what credentials the automation holds against your systems, where intermediate results are stored, how long they persist, which vendor staff can access the running system, and what happens on failure.

The question that separates the two usefully is: what could a compromised account at the vendor reach? If the answer is "the credentials for our document management system, and the queue containing the last thirty days of extracted content," you now know where the review should concentrate. A zero-retention agreement with a model provider does nothing about that exposure.

The access question

Ask what credentials the workflow needs and why. Then push on scope.

The right answer is a dedicated service account with the narrowest permissions that let the workflow function: read access limited to the specific objects it processes, write access limited to the specific destination, and nothing else. Not a shared account. Not an admin credential because scoping was fiddly. Not a human user's credentials because that was faster to arrange.

Related questions worth asking explicitly:

• How are those credentials stored, and who at the vendor can retrieve them?
• Can they be rotated without redeploying the workflow, and has that been tested?
• What happens to access when the engagement ends, and how quickly?
• Is there a break-glass path for vendor staff into the running system, and is its use logged and alerted?

"We would need broad access to make this simpler" is a legitimate answer to receive, and a legitimate reason to negotiate. It is not a legitimate reason to skip the question.

Retention, at every hop

Retention is usually stated once, for the primary datastore, and left unstated everywhere else. Ask for it hop by hop, using the diagram from the first step.

For each place data comes to rest: how long does it persist, what deletes it, and is that deletion verified or assumed? Backups deserve their own answer, because "deleted after 30 days" often means deleted from the live system and retained in backups for a year, which may be entirely fine but should be a decision rather than a surprise.

Error and exception paths deserve particular attention. Data that flows cleanly through a workflow may be retained for minutes. Data that triggers an error frequently ends up in a dead-letter queue or an error log with no retention policy at all, and error cases are disproportionately likely to involve the unusual documents you care most about.

What the human review layer means for risk

Most well-designed AI workflows keep a human in the loop for low-confidence cases. This is good for accuracy, and it introduces a control point worth understanding in a security review.

Ask where that review happens. If exceptions surface inside your own systems, the reviewer is your employee under your access controls and the exposure is contained. If exceptions surface in a vendor-hosted console, then vendor staff and your reviewers are looking at the same interface, and the access model for that console is now part of your data protection posture.

Neither design is wrong. But the second one needs the same scrutiny as the workflow itself, and it is frequently omitted from the review entirely because it is thought of as a product feature rather than a data path.

Auditability, decided before go-live

The requirement to state up front: every transaction produces a record showing what came in, what the system extracted or decided, what confidence it attached, whether a human reviewed it, and what was written downstream.

This is straightforward to build in from the start and painful to retrofit, because it changes the data model. Without it you cannot investigate an incident, cannot explain an incorrect downstream record, and cannot demonstrate to an auditor that the control you described is the control that ran.

Make it a requirement in the statement of work rather than a question in the security review. By the time the security review happens, the architecture is usually already fixed.

The questions that separate serious vendors

A handful of questions reliably distinguish vendors who have deployed into regulated environments from vendors who have not:

• "What happens when the model is confident and wrong?" Anyone can describe the low-confidence path. The dangerous case is high confidence on a wrong answer, and a vendor who has run real volume has thought about detecting it downstream.
• "Show me the audit record for a single transaction." Not the schema. An actual example, redacted.
• "What is in your logs right now?" The gap between what a vendor intends to log and what the application actually logs is frequently large.
• "Who at your company can see our data, and what would that access look like in your logs?"
• "What is your process when a model provider changes a model version?" Behaviour shifts between versions, and a vendor without an evaluation process for that is going to discover changes through incidents rather than testing.

Vague answers here are more informative than any certificate. The point of the exercise is not to collect documents; it is to find out whether the people building your workflow have thought about failure.

Bring the review forward

The recurring theme across all of this is timing. Most of what matters, the access model, the retention design, the audit record, the location of the review interface, is decided during architecture and expensive to change afterwards. A security review that runs at contract signature is reviewing decisions that have already been made.

The better pattern is a short conversation about the data path during scoping, well before anything is built, so that the security requirements shape the design rather than grade it. For teams working through this on their own systems, our enterprise page covers how we approach deployments where this scrutiny is the norm, and the security overview document goes through our own posture in detail.

It is also worth remembering that the same discipline pays off operationally. The controls above, narrow access, per-transaction logging, explicit retention, are the same controls that make a workflow debuggable when it drifts. Security review done early is not a tax on the project. It is most of what separates an automation that survives its second year from one that quietly stops being trusted.