The Creative Scoring Model Our Team Actually Uses (Not a Black Box)14

The Creative Scoring Model Our Team Actually Uses

Most teams treat "creative quality" as a vibe. Someone reads the output, shrugs, and writes "good" or "meh" in the doc. We wanted something better: a scoring model that is transparent, auditable, and fast to compute. Not a black box. Not a 40-dimensional embedding you need a GPU to decode. A model you can explain to a product manager over coffee.

Here's the one we actually use.

The Core Idea: Five Axes, One Number

We score creative output along five axes. Each axis is a single scalar in $[0, 10]$. The final score is a weighted sum:

$$

S = \sum_{i=1}^{5} w_i \cdot s_i, \quad \sum w_i = 1

$$

The weights we use in production:

Originality        0.25
Coherence          0.25
Resonance          0.20
Surprise           0.15
Efficiency         0.15

Why these five? Because they cover the space where "creative" actually lives. Let me unpack each.

1. Originality — "Has this been seen before?"

Not "is it new" in some absolute sense. Nothing is truly new. Originality here means: does this combine elements in a way the audience hasn't recently encountered?

We score it by asking a simple question: If I showed this to three domain experts, would at least two say "I haven't seen that before"?

• 10: Genuinely novel structure or concept

• 7: Familiar ingredients, unusual combination

• 4: Competent but expected

• 1: Could be a template fill-in

Practical tip: Keep a running "reference shelf" — a folder of the 50 most common patterns in your domain. If your output closely matches any of them, originality drops.

2. Coherence — "Does it hang together?"

This is the anti-fragmentation axis. A creative piece that is a collage of five unrelated ideas scores low here, no matter how clever each piece is in isolation.

$$

C = \frac{\text{number of logical transitions that work}}{\text{total transitions}}

$$

We don't count transitions mechanically. We read the piece once and ask: Where did I feel a seam? Each seam costs 1 point. Start at 10, subtract.

• 10: Reads as one unified thought

• 7: One or two seams, but the overall arc holds

• 4: Feels assembled

• 1: Feels like a playlist, not a song

Why it matters: This is the axis where LLMs are strongest and humans are weakest. Machines are coherent; humans are coherent. Both. But when you use an LLM to generate and a human to edit, you want to verify coherence didn't survive the edit.

3. Resonance — "Does it land?"

This is the subjective axis, and I'll be honest: it's the hardest to calibrate. Resonance means emotional or intellectual stickiness. Does the reader/listener/viewer come away with a feeling or a thought they hadn't had before?

We use a proxy: the 3-second test. After first exposure, close the document. Wait three seconds. What's the one thing you can recall? If the answer is "I don't know," resonance is low. If you can articulate a specific line, image, or idea, resonance is high.

• 10: You can teach someone else the core insight in one sentence

• 7: You remember the general flavor

• 4: You remember it existed

• 1: You've forgotten it

Caveat: Resonance is audience-dependent. We calibrate our scoring team to a target audience. A PhD paper scores differently than a blog post, and that's fine. The model is relative to the context.

4. Surprise — "Did it go somewhere I wasn't expecting?"

Surprise is not the same as originality. Originality is about the ingredients; surprise is about the trajectory. A piece can be made of completely familiar elements but take an unexpected turn.

$$

S_{surprise} = 10 \cdot \frac{\text{number of "wait, what?" moments}}{1}

$$

We cap it: more than two genuine surprises in a short piece starts to feel gimmicky. One or two well-placed surprises score 8–10. Zero scores 3. Three or more scores 5 (diminishing returns).

The key distinction: A surprise is a rewarded expectation violation. If the reader is confused rather than delighted, it's a plot hole, not a surprise. This axis punishes incoherent surprise.

5. Efficiency — "Is every element earning its place?"

The anti-bloat axis. Creative work often over-explains, over-decorates, or includes elements that don't serve the core idea. Efficiency asks: If I removed this sentence/image/line, would the piece be weaker or roughly the same?

• 10: Remove any element and it gets worse

• 7: 80% of elements are load-bearing

• 4: Half the piece could be cut

• 1: Decorative excess

This is where AI-assisted work often shines (LLMs tend to be efficient) and where human creative work often struggles (we love a good tangent).

Putting It All Together

Here's a worked example. A 200-word product narrative:

A 7.05/10 is a solid B+. It's coherent, efficient, and reasonably original, but the resonance and surprise are mediocre. The fix is clear: sharpen the core insight (resonance) and add one well-placed unexpected turn (surprise).

Why Not a Black Box?

You could train a neural network to predict creative quality from text. You could use a 768-dimensional embedding and compute cosine similarity to a "good creative work" centroid. You could do all of that.

But then you can't explain why the score is what it is. You can't tell a client "your piece scored 6.2 because the coherence axis was dragging." You can't train a junior writer on what to improve. You can't audit whether the model is biased against a particular genre.

Five axes. Five questions. One weighted sum. A junior analyst can apply it. A client can understand it. A PM can argue with it.

That's the point. A scoring model you can explain is a scoring model you can improve. A black box you can only calibrate by throwing more data at it.

One Last Thing: The Model Is a Conversation

These weights are not sacred. For a brand voice that prizes novelty, bump originality to 0.30 and drop efficiency to 0.10. For a technical whitepaper, bump coherence and resonance. The axes stay the same; the weights flex.

And the axes themselves? We revisit them every six months. Last year we considered adding "specificity" as a sixth axis. We ran it for two months, found it correlated 0.91 with resonance, and folded it in. If an axis doesn't differentiate — if everything scores 6 or 7 — it's not doing its job.

That's the whole philosophy. Five questions. One number. A model you can argue with. Not a black box. A tool.

— Dr. Lena Vasquez, AI Research