Validation looks simple from the outside: receive a submission, check it against the rules, accept or reject.
The reality, at scheme scale, is that 80% of the work isn't the validation. It's the exception handling, the audit trail, the explanation back to the submitter, and the routing of edge cases to the right human for the right reason at the right time.
Digital Experience Labs has built this discipline across ANZ: for a state insurance regulator (lodgement validation at scheme scale), a state government department (training contract management), and across regulated insurance contexts where product configuration validation has to happen before launch. The lesson, repeated across every engagement: real-time validation is achievable, but only when you treat the exception path as a first-class design concern, not an afterthought.
The defining quality of a good large-scale validation system is not what happens when the rule passes. It's what happens when it doesn't.
What validation actually means
Validation gets confused with three adjacent concepts. Worth being precise, because conflating them in design causes real architectural problems:
- Verification proves identity or authenticity: is this person who they say they are; is this document genuine?
- Validation checks a submission against a defined ruleset: does this lodgement meet the scheme's requirements?
- Approval grants permission: is this entity now allowed to proceed?
- Eligibility qualifies for a programme: does this applicant qualify?
Most regulatory and government B2B systems combine all four. Validation is typically the highest-volume layer, which is why it's where automation has the most leverage.
When real-time matters, and when it doesn't
Real-time validation is essential when submissions are interactive (a citizen filling in a form expects a result while they're still on the page), when downstream processes can't proceed without a decision (a regulator processing lodgements at scheme scale, where queuing compounds), or when batch latency creates a backlog that builds overnight.
Real-time is the wrong answer when the workload is purely back-office with no human waiting, when validation needs data the engine doesn't have synchronously, or when the volume doesn't justify the complexity.
Latency is a cost, not a virtue. Design for the SLA you actually need.
Three patterns we have shipped
1. Scheme-level lodgement validation
Millions of submissions per scheme, each validated against a regulatory ruleset, with exception escalation and explanation back to the submitter. The validation engine handles the routine work; the operational team handles edge cases with full context already gathered by the engine.
The key design decision wasn't the rules. It was the explanation back to the submitter when a rule failed, specific enough to be actionable ("Field 7.3 requires a value matching pattern X, and you submitted Y"), not just "validation failed." And the escalation path for ambiguous cases, where the ruleset doesn't cleanly say yes or no.
Where we've shipped this: a state insurance regulator in Australia. Lodgement validation at scheme scale, with full audit trail and point-in-time replay.
2. B2B government contract validation
A government department managing training contracts across hundreds of registered providers. Each contract validated against eligibility, compliance, audit, and regulatory rules before approval. The validation engine surfaces exceptions with the specific rule that failed; the operational team adjudicates with full context.
The operational challenge here wasn't volume. It was the constant change in the ruleset. Regulatory requirements shifted quarterly. Provider eligibility criteria evolved. The validation engine had to absorb rule changes without downtime, and every decision had to be reproducible against the rules-in-force at the time it was made.
Where we've shipped this: a state government department in Australia. Training contract management across education and workforce development.
3. Insurance product and quote validation
Two related patterns. First: before a newly-configured insurance product can launch, it has to validate against internal underwriting rules, regulatory constraints, pricing parameters, and operational compatibility. This is rating's mirror, instead of validating per-quote, you validate the product definition itself, preventing downstream failures across millions of quotes.
Second: real-time validation of a quote against eligibility, pricing, and underwriting rules before binding. The agent or system is waiting; the validation has to be quick, complete, and explained.
Where we've shipped this: Solution Underwriting (now CFC) and Envest, across specialty underwriting and general insurance in ANZ, the same discipline applied at different points in the product lifecycle.
The exception is the product
Most validation systems we encounter were built for the happy path. The rule passes, the submission proceeds, everyone moves on. Exception handling grew organically: a catch-all queue, a generic error message, a reviewer who eyeballs everything that failed.
The good validation systems design the exception path first, then build the rules around it:
- What's the explanation back to the submitter? "Validation failed" isn't an explanation. The submitter needs to know exactly what failed and exactly what to fix.
- What's the escalation path? Who reviews the exception, with what context, against what SLA?
- How does the submitter resubmit? Can they edit just the failing field, or do they start over?
- What's the audit trail of the exception? Months later, who decided what, when, and why?
Get the exception path right and adoption follows. Get it wrong and you've built friction: every submitter learns to dread the system, and support costs eat whatever you saved on automation.
Governance and audit
In a regulated environment, the validation system has to answer two questions:
- What was the decision? Routine. Every validation system tracks this.
- What would the decision have been six months ago, given the rules in force at the time? Much harder.
Question 2 requires versioned rule storage, immutable decision logs, and point-in-time replay. Government and insurance regulators ask it routinely; for public-sector services, the Australian Government's Digital Service Standard sets the broader expectation. If you can't answer it, you've built a tactical system, not a strategic one.
How Digital Experience Labs approaches it
Discovery starts with the exception path: what currently fails, where it routes, how long it takes, and what the submitter experiences. We've walked into engagements where the exception queue was the primary workload, where the system was generating more exceptions than clean passes because the rules were written for a simpler era and the submissions had outgrown them.
We build the rule platform to handle the happy path quickly, then design the exception path explicitly: who reviews, with what context, against what SLA, with what audit trail. The validation engine is the easy part. The operational model around it is the engagement, and it's the same discipline behind our wider process and decision automation work.
Frequently asked questions
What's the difference between validation, verification, and approval?
Validation checks a submission against a defined ruleset. Verification proves identity or authenticity. Approval grants permission. Eligibility qualifies for a programme. Most government B2B and regulated insurance systems combine all four, but they shouldn't be conflated in design; the architectural concerns differ.
When should validation be real-time vs batch?
Real-time when a human or upstream system is waiting, or when batch latency creates compounding queues. Batch when the workload is back-office with relaxed SLAs. Most workloads need both, with a single source of truth for the rules.
How do you validate insurance product configurations across thousands of variants?
Validate the product definition itself, not each quote. Check the configuration against underwriting rules, regulatory constraints, pricing parameters, and operational compatibility before the product goes live. This prevents downstream failures across millions of quotes.
What scale do real-time validation services need to operate at?
Interactive citizen-facing flows: tens to hundreds per second at peak, sub-second response. Machine-to-machine B2B: thousands per second. Scheme-level lodgement: millions per month, concentrated in submission windows.
How do you handle exceptions in high-volume validation flows?
Design the exception path before the happy path. Capture the specific failing rule, the data that caused it, and the context the reviewer needs. Route to the right queue by rule category, not just "validation failed." Make the explanation to the submitter specific and actionable. Track SLA on resolution.
What's the audit-trail requirement for government B2B validation?
Every decision reproducible months later with the rules-in-force at the time. Versioned rule storage, immutable decision logs, point-in-time replay. This is a foundational architectural decision, not a feature you bolt on later.
How do you avoid validation becoming a bottleneck?
Two patterns. First: separate routine validation from exception adjudication; routine should be sub-second and automated, exceptions get the time they need. Second: design exceptions to require minimum reviewer effort; the engine should have already gathered all relevant context.
Can business rules and AI co-exist in a validation pipeline?
Yes, and they should. AI handles semantic interpretation: reading documents, classifying intent, extracting structured data from unstructured input. Rules handle deterministic decisioning: given clean inputs, was this submission valid? AI prepares the input, rules govern the decision, audit covers both. We unpack this split in the Decision Pyramid.
Where to next
Scaling or replacing a validation workload?
If you're scaling a validation workload, or replacing a legacy system that's hitting its limits, we'd happily walk through the operational design questions before the technology questions.
Rating engines are the mirror pattern, same discipline, different shape; we've written about that separately. Going deeper by sector: Government and Insurance.