Tenders often ask for "ANPR accuracy above X%". Vendors answer with one number. Unless both sides agree on the definition, the number cannot be compared or checked. This post sets out the definitions we use, how we split results, and how large a test must be.
Key points
- Detection rate counts vehicles whose plate was found. Read rate counts found plates that were read correctly. End-to-end accuracy counts all vehicles read correctly.
- Say who decides that a plate is "readable", because that choice sets the denominator.
- Report day and night, speed bands and vehicle classes separately.
- On 500 vehicles, a measured 96% has a 95% confidence interval of about 93.9% to 97.4%.
- Keep the test data apart from the training data, and keep the evidence images.
Three metrics, three denominators
| Metric | Numerator | Denominator |
|---|---|---|
| Vehicle detection rate | Vehicles detected | Vehicles that passed (ground-truth count) |
| Plate detection rate | Vehicles with a plate found | Vehicles that passed with a plate |
| Read rate (exact match) | Plates read with every character correct | Plates found |
| End-to-end accuracy | Vehicles with the plate read exactly right | Vehicles that passed with a plate |
| Character accuracy | Characters correct | Characters in all plates found |
End-to-end accuracy is roughly plate detection rate × read rate. It is the number an operator feels. A system can report 99% detection and 95% reads, and still read only about 94% of vehicles correctly.
We follow this rule in our own figures. The Recon Global Shutter Camera FAQ states plate recognition accuracy of 96% for readable number plates, day and night, and a vehicle detection rate above 99.9%, and adds that these refer to different metrics and should not be combined into a single headline number.
Character accuracy flatters
Character accuracy is always higher than plate accuracy. If errors were independent across a 10-character plate:
plate accuracy = character accuracy ^ characters
0.99 ^ 10 = 0.904
0.995 ^ 10 = 0.951
0.98 ^ 10 = 0.817A plate with one wrong character is a wrong plate for enforcement and access control. Report exact-match plate accuracy, and give character accuracy only as a diagnostic.
Who decides "readable"?
Many figures, including ours, are stated for readable plates. The definition must be written down before the test: for example, a plate that a trained person can read with certainty from the evidence image. Plates that are missing, fully covered or broken beyond reading are then counted separately, not dropped silently. Report how many plates were excluded and why. A test that excludes 20% of vehicles as unreadable measures something different from one that excludes 2%.
Split by condition
One number across all conditions hides the hard cases. We recommend reporting at least these bands:
- Light: day, night, dawn and dusk.
- Weather: dry, rain, fog.
- Speed: for example below 40 km/h, 40 to 80 km/h, 80 to 120 km/h and above 120 km/h, from an independent speed source.
- Vehicle class: two-wheeler, three-wheeler, car, bus, truck.
- Plate type: HSRP, non-HSRP, BH series, EV, commercial, two-line, hand-painted.
Vehicle class matters most in India. GateGuard publishes above 98% for four-wheelers and above 92% for two-wheelers as two numbers. Blending them would make the result depend on the traffic mix of the test site. Our post on reading Indian number plates explains why two-wheelers are harder.
How many vehicles do you need?
An accuracy measured on a sample is an estimate. Its uncertainty depends on the sample size. The Wilson score interval, described in the NIST/SEMATECH e-Handbook, works well for proportions near 100%:
from math import sqrt
def wilson(correct, n, z=1.96):
p = correct / n
d = 1 + z * z / n
centre = (p + z * z / (2 * n)) / d
half = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return centre - half, centre + half| Vehicles tested | Measured accuracy | 95% interval |
|---|---|---|
| 100 | 96% | 90.2% to 98.4% |
| 500 | 96% | 93.9% to 97.4% |
| 1,000 | 96% | 94.6% to 97.0% |
| 5,000 | 96% | 95.4% to 96.5% |
A 100-vehicle trial cannot tell 92% from 98%. Each condition band needs its own sample, so a test with eight bands needs many more vehicles than a single headline would. Vehicles passing in one platoon are not fully independent either, so spread the test across days and times.
Keep the test honest
- Ground truth: count vehicles independently of the system under test, for example by manual review of continuous video.
- Hold-out data: test on corridors and times the model was not trained on. Retraining on test data invents accuracy.
- Evidence: keep the evidence image for every read. A correct read without a usable image is incomplete for enforcement.
- Publish the method: metric, denominator, exclusions, bands, sample sizes and dates.
Speed bands need an independent speed reference; our post on radar and camera fusion describes how speed is measured. Capture quality at speed is in our global shutter post. Our published figures are on the Recon Global Shutter Camera and Lane ANPR pages, and our methodology brief is on the Research page.
Frequently asked questions
What should a tender ask for?
Ask for end-to-end exact-match accuracy on all vehicles with a plate, split by day and night, vehicle class and speed band, with the definition of "readable", the exclusion count and the sample size per band.
Is 99.9% vehicle detection the same as 99.9% ANPR accuracy?
No. Detection says a vehicle was found. It says nothing about whether its plate was read correctly.
How long should a field trial run?
Long enough to cover day and night, the weather you expect and each speed band with enough vehicles per band. Use the interval table above to choose the count.
Share this article