Reason or Recite, explained: our ECCV 2026 workshop paper on traffic anomaly reasoning

The paper is "Reason or Recite: Complementary Specialists with an LLM Judge for Traffic Anomaly Reasoning". It was accepted as an oral paper at the 10th AI City Challenge Workshop at ECCV 2026 in Malmö, Sweden. This post explains the problem, the three parts of the method, the results and where to find the code.

Key points

• The TAR benchmark scores two kinds of question in two different ways: exact match for yes/no and multiple choice, and BERTScore against one reference text for free-text answers.
• We found that these two scoring rules reward different behaviour, so we trained two specialists instead of one model.
• A 9B reasoning specialist learns from rewritten, timestamped reasoning traces. A 4-bit 27B answer specialist learns from the gold answers only.
• A frozen 27B judge, constrained against its own position bias, selects one candidate per question.
• Result: first of 27 teams on the public TAR leaderboard, with 1.000 on binary event verification.

Figure 2 of the paper. An official reasoning trace (left) and its TAR-RE rewrite (right). The gold answer stays fixed.

The task: traffic anomaly reasoning

Track 3 of the 2026 AI City Challenge asks one video-language model to answer ten task types about CCTV traffic footage. Nine task types are scored. The organisers excluded temporal localisation from the score.

The training set has 44,040 annotations over 3,670 CCTV videos (about 26.1 hours) from eight public datasets. The in-domain test set has 960 human-verified annotations over 80 clips. The leaderboard mean is the unweighted average over the nine scored task types. The dataset is on Hugging Face as nvidia/PhysicalAI-Traffic-Anomaly-Reasoning.

Why one model is not enough

Closed-form questions reward precise reasoning that is grounded in what the video shows. Open-ended answers are scored by similarity to one reference text. The training references were written by an automatic labelling pipeline, so matching their style matters as much as being correct. A model that reasons better than its supervision can score worse on those tasks.

We therefore split the two objectives across two models. We call them the reasoning specialist and the answer specialist. Our post on why BERTScore can punish better answers shows two examples in detail.

Part 1: TAR-RE, rewriting the reasoning traces

Before we trained anything, we audited the training annotations. We found four defect classes: reasoning that cites captions the model never sees, four or more timestamp formats, labels that contradict the video, and verdicts with no timestamp. The full audit is in our post on auditing the labels.

TAR-RE (TAR Reasoning Enhanced) repairs the supervision in two steps:

• Normalise time: every timestamp becomes one textual form, <X.Y seconds>. This matches the textual timestamps that Qwen3-VL interleaves with video tokens.
• Regenerate the reasoning: Qwen 3.5 9B in thinking mode rewrites every trace into five sections: scene, query, timestamped evidence, reason and conclusion.

Two constraints keep the rewrite honest. The gold outcome is frozen, so the rewrite cannot relabel the data. The gold wording is banned, and a validator rejects any trace that misses a tag, misses the depth floors (220 words for closed-form tasks, 320 for descriptive tasks) or contains the gold answer verbatim.

Part 2: two specialists

The reasoning specialist verifies before it answers. On a subset of our internal test split, trace supervision lifts multiple choice from 0.9429 to 1.0000 and binary verification from 0.9571 to 0.9714. The answer specialist absorbs the reference style and wins the free-text tasks on average. The training details are in our post on fine-tuning a 27B model on one GPU.

Part 3: a judge that picks per question

For every test question, a frozen Qwen 3.6 27B receives the video frames, the question and both candidate answers. It returns which answer to submit. It has no fine-tuning and no routing table.

An unconstrained judge did not work. Under a neutral rubric it favoured the last-shown answer on 65 to 79% of items, and only 51.6% of its verdicts survived an order swap. We therefore present the candidate that validation favours last, and the judge keeps it unless the video clearly contradicts it. The details are in our post on judge bias.

Results on the official leaderboard

The official AI City Challenge 2026 winners page lists the TAR leaderboard as follows.

The table in our paper does not include UOB&UW Team, so we use the official page as the reference. The margin over the runner-up is 0.0009. The challenge overview paper (arXiv:2608.17044) reports our rank as first of 27 teams on the public leaderboard and first of 76 on the general leaderboard.

Our submission scored 1.000 on binary event verification, 0.950 on multiple choice and 0.669 on event verification with explanation. It had the best score on four of the nine task types and stayed within 0.025 of the best on every other task.

What we take from it

• Audit auto-labelled data before training on it. The defects shaped every design decision.
• When a metric scores similarity to one reference, reasoning quality and reference matching are separate objectives.
• A judge model needs controls for position and self-preference bias before it can add value.

We build vision language models into our products as well. AI Search runs a vision language model on the Recon camera for natural-language video search. More of our work is on the Research page and in our press release on the result.

Paper, code and citation

• Paper: arXiv:2610.34993 (18 pages, 3 figures, 5 tables)
• Reviews and venue record: OpenReview
• Code and the exact leaderboard submission: github.com/mayur-ag/reason-or-recite
• 9B reasoning specialist weights: Hugging Face
• Workshop programme: AI City Challenge 2026 workshop

@inproceedings{kulkarni2026reason,
  title     = {Reason or Recite: Complementary Specialists with an {LLM} Judge for Traffic Anomaly Reasoning},
  author    = {Kulkarni, Mayur},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
  year      = {2026}
}

Frequently asked questions

Is this an ECCV main conference paper?

No. It is a workshop paper. It was accepted as an oral presentation at the 10th AI City Challenge Workshop, held with ECCV 2026 in Malmö.

Which models does Reason or Recite use?

The reasoning specialist is Qwen 3.5 9B with LoRA. The answer specialist is Qwen 3.6 27B with QLoRA. The judge is a frozen, off-the-shelf Qwen 3.6 27B.

Can the result be reproduced?

The repository holds a six-step driver and the exact submission file. Greedy decoding with vLLM is not fully deterministic, so 5 to 10 items can switch between candidates from run to run.