18 September 2026 · 1 min read · Humza Imran

Why evaluation is the hard part of applied AI

The model is rarely the bottleneckMost projects fail on evaluation, not architecture.Define the decision firstBuild the test set before the model

The model is rarely the bottleneck

Most projects fail on evaluation, not architecture.

  • Define the decision first
  • Build the test set before the model

Read next

17 Sep 2026

Your Model Is Not the Product

The model is perhaps a tenth of what you are shipping. The other nine tenths decide whether anyone keeps using it.

6 min read · 11 views

10 Sep 2026

Evaluation Sets That Lie to You

A held-out set is only held out if it was separated before you started making decisions. Usually it was not.

8 min read · 6 views

1 Sep 2026

Retrieval Is Mostly Chunking

Swapping embedding models buys you a little. Getting chunking right buys you a lot. The attention goes the wrong way round.

7 min read · 6 views

0 comments

No comments yet. Be the first.

Leave a comment

Never published.

Plain text only. Comments from visitors are reviewed before they appear.