What to automate in a regulated operation

Most automation projects fail before a line of code is written, because the wrong process was chosen. In a regulated financial operation the cost of choosing wrong is higher than usual: automate the wrong thing and you have not saved anyone time, you have created a compliance problem.

I spent months working a corporate onboarding process and a support queue by hand before any of it was automated. That order matters. You cannot see which parts of a workflow are actually repetitive from a process diagram — you see it from having done the work two hundred times and noticing that you check the same three systems in the same order every time.

Here is the test I ended up using.

Three axes, not one

Most people score a process on one axis: how much time it takes. That is the wrong question, because a slow process that happens twice a month is worth less than a fast one that happens two hundred times a day, and a high-volume process that is different every time cannot be automated at all.

Score it on three.

Volume. How often does this actually happen? Not how long it takes — how many times. A workflow that runs constantly is worth automating even if each instance is quick, because the saving compounds and because the automation gets enough traffic to reveal its own failure modes quickly.

Standardisation. How much of it follows repeatable rules? A deposit investigation is highly standardised: identify the transaction, check its status, check confirmations, explain the result. A dispute over an account closure is not — every one is different, and the parts that matter are the parts that vary.

Risk. What happens when it is wrong? This is the axis people skip, and it is the one that decides the shape of the solution rather than whether to build it.

What the three axes tell you

The combination, not any single score, tells you what to build.

High volume, high standardisation, high risk. Automate the preparation, never the decision. Corporate onboarding is the clearest example: reading incorporation documents, extracting directors and shareholders, tracing ownership up through corporate parents to the ultimate beneficial owners, and noticing which documents are missing or expired is all mechanical work that a person should not be doing by hand. Deciding whether to onboard that company is not. So the system assembles the dossier and a compliance officer decides. It recommends; it does not approve.

High volume, high standardisation, low risk. Automate it end to end. Corpus work sits here: finding, extracting and deduplicating source material, stripping boilerplate and secrets, scoring what is dense enough to keep. Nobody needs to approve a discarded duplicate, and the volume is far past what a person can read.

Anything genuinely varied. Do not automate it — improve the process first, or accept that it is human work. Trying to make a model handle real variety produces something that is right most of the time, which in operations is another way of saying it fails unpredictably.

Low volume, anything. Leave it. The maintenance cost of an automation nobody exercises exceeds the work it saves, and it will be quietly broken the first time you actually need it.

The boundary is the design

The interesting part of an operational system is not what it does. It is what it refuses to do.

Wherever the risk score is high, there has to be a hard structural rule about where the machine stops — not a soft preference. A document pipeline that traces ownership and flags an expired certificate must not also decide whether to onboard the company. A system answering questions about money must hand back whenever the answer would move funds, the situation looks security-sensitive, the state is genuinely unclear rather than merely pending, or the request falls outside what it was built for.

The rule has to be structural because a model instructed to "stop if unsure" will occasionally be confidently wrong, which is exactly the case the rule exists to catch. That boundary should be the one edge in the architecture that cannot be optimised away.

And stopping is not failure — it is a handoff, and it should arrive prepared. The system should hand the human a package: the case, the evidence it gathered, a classification, what it already checked, what it found, and a short summary. Done well, the handoff still saves the reviewer most of the work even though nothing was resolved automatically.

The same logic applies to the attack surface. A system customers can type into will be probed — for prompt injection, for actions outside its authority, for whatever internal data it can be talked into repeating. Those defences belong in the design from the start, and they need testing against deliberate misuse rather than happy paths.

How you know it works

The part that surprised me: the hardest problem was never the model. It was knowing whether the thing was actually right.

Automation in a regulated operation cannot be signed off on a demo. Every system I have worked on needed its own definition of correct, and a repeatable way to check it after the next change:

• Retrieval systems need to be judged on whether the answer is grounded in what was retrieved, not on whether it reads well. A fluent wrong answer is worse than an obvious one.
• Extraction pipelines need a labelled set where you already know the right directors, the right ownership percentages, the right expiry dates — and you measure against it.
• Generative systems need a numeric gate. On identity-preserving image work I scored generated output against reference embeddings with ArcFace cosine similarity and set an acceptance threshold, plus CLIP alignment for whether the scene matched the prompt, plus blind A/B against a written defect checklist. The threshold was project-specific, not universal — the point is that there was one, and it was applied the same way every time.
• Anything customer-facing needs adversarial testing, not just functional testing. You are checking what a determined user can make it do.

None of that is glamorous, and it is the difference between a system you can put in front of real users and a demo that impressed someone in a meeting.

A validator can teach a model to lie

The sharpest thing I have learned since writing the list above is that a check can be worse than no check.

On an ownership pipeline, the validation rule was the obvious one: the declared shareholdings in a document should sum to 100%. It is correct arithmetic and it reads like diligence. But documents are incomplete — a register lists four shareholders and omits the fifth, or publishes a band rather than a number. So the rule fails on a truthful extraction, and the only way for the model to pass is to invent the missing shareholder.

The validator was not detecting fabrication. It was paying for it. And because the schema still validated, every dashboard reported success.

The fix was to stop demanding a false precision the source never had: check that the declared holdings are consistent with what the document says, as intervals, and let an incomplete document remain visibly incomplete. Which generalises past this one system — if your validation rule can only be satisfied by information the source does not contain, you have not written a test. You have written an incentive.

The arithmetic is not the easy part

One more correction to my own earlier assumption. I used to think the modelling in this work was upstream, in the extraction, and that once you had the ownership percentages the rest was multiplication.

It is not. Ownership loops — companies hold each other, directly or through a chain — and the moment it does, multiplying along the chain silently under-reports the real holding. The correct treatment is an input-output matrix inversion, and the gap between the two is not a rounding error: on the reference case I work with, a 4.24% direct stake is a 91.81% effective one. A 25% threshold sits between those two answers, so the naive method does not merely lose precision, it returns the opposite regulatory determination.

The lesson I would keep is the shape of it rather than the linear algebra. When the output of an AI system feeds a threshold, the arithmetic after the model deserves the same scrutiny as the model — and it is far more likely to be wrong quietly, because nobody thinks to test it.

What I would tell someone starting

Work the process manually first, long enough to be bored by it. Boredom is the signal — the parts that bore you are the parts that are standardised enough to automate.

Then score honestly on all three axes, and let the risk score decide the shape. Full automation, prepared-decision, or human-only are three different products, and picking the wrong one is not something better prompts can fix later.