I Let AI Decide Which Ads to Kill — Revenue Jumped 47% in Two Weeks
I Let AI Decide Which Ads to Kill — Revenue Jumped 47% in Two Weeks
The $200,000 Question No One Was Asking
Most ad managers optimize for the ads that work. I spent two weeks optimizing for the ads that shouldn't have survived.
Here's the thing about performance marketing that nobody tells you when you're starting out: killing bad ads is harder than finding good ones. Finding a winning creative is a search problem — you test, iterate, narrow down. Killing a losing one is a judgment call, and judgment calls are where humans get emotionally attached to their own work.
I was managing a $200,000 monthly ad spend across 14 campaigns, 312 active ad creatives, and four platforms. Every Tuesday morning, I'd open the dashboard and do the same ritual: sort by spend, look at ROAS, and make a gut-feel call on which 15-20 ads to pause. I'd keep the ones that "felt" like they were on a trend, give the underperformers another day because "maybe tomorrow," and rationalize the expensive losers because "the creative is good, the audience is just wrong."
The average ad in my portfolio lived for 11.4 days. Of those, 68% spent at least 7 of those days underperforming the portfolio median. I was keeping 68% of my ads alive past the point where they deserved to die.
That's not optimization. That's sentiment.
What I Actually Built (It's Simpler Than You Think)
I'm not going to pretend I built a custom GPT-4 agent with a 400-line Python pipeline. I didn't. What I built is a structured evaluation framework that feeds weekly performance data into an LLM and asks it to make keep/kill/observe decisions with justification.
The data I pull every Monday:
Spend, impressions, clicks, conversions, CPA, ROAS per creative over the trailing 7 days
Same metrics over the trailing 28 days (so I can see trend direction, not just a single week)
Creative age in days
Platform, campaign, audience segment
CTR and CPC relative to the campaign median (normalized)
I feed all of that into a prompt that asks the model to score each creative on four axes:
Efficiency — Is the CPA below, at, or above the campaign target?
Trend — Is the metric improving, flat, or degrading over the 28-day window?
Exposure — Has this creative seen enough impressions to be statistically reliable? (I set a floor: under 15,000 impressions = "too early to judge")
Substitution — Is there a newer creative in the same campaign that's outperforming it? If so, the older one is a candidate for retirement even if it's not "bad."
The model returns a table: creative ID, score (1-10), decision (KEEP / OBSERVE / KILL), and a one-sentence justification.
I review the list. I override maybe 8-10% of the kills (usually because I know a new variant is launching or a product price change is mid-flight). I don't fight the model on the rest.
Total setup time: about 4 hours. Total review time per week: 25 minutes.
The 47% Number and How to Read It
Revenue in week 1 of the experiment: $184,300. Week 2: $270,800. That's a 46.9% increase.
But I want to be honest about what's in that number and what isn't. The 47% doesn't mean the AI "generated" 47% more revenue. It means:
I killed 41 ads in week 1 that were burning budget at ROAS below 1.2. That freed up roughly $18,400 in spend that got reallocated to the top-performing 20% of creatives.
I killed 29 more in week 2 after the first round of reallocation showed which "kept" ads were actually plateauing.
The budget that used to feed the 33% of ads that were quietly bleeding now fed the 20% that were compounding.
The math: if 68% of ads were underperforming and I killed 55% of those (being conservative, not killing everything the model flagged), I reallocated roughly 30% of my total spend from low-ROAS to high-ROAS creative. At a blended ROAS of 2.1 on the winners and 1.1 on the losers I killed, that's a net revenue lift of roughly 40-50%.
So 47% is a realistic, not a miraculous, number. It's the arithmetic of stopping a slow bleed.
Where the AI Was Right and Where It Wasn't
This is the part most "AI for marketing" articles skip. The model was great at the mechanical parts:
Statistical reliability: It correctly flagged 12 creatives I'd been keeping alive that had under 15,000 impressions and were "promising" on 3,000 impressions of data. I'd be killing them at 40,000 impressions if I'd let the model decide. It told me to wait. Good call.
Substitution detection: It caught 6 older creatives that were still at ROAS 1.8 but had been outperformed 2:1 by a newer creative in the same campaign. I would have kept them for another week of sentiment.
Trend direction: It correctly identified 4 creatives whose 7-day ROAS was 3.1 but whose 28-day trend was degrading from 4.2 to 3.1. I would have celebrated the 3.1. It told me to start preparing a replacement.
Where it was less sharp:
Creative quality judgment: It couldn't tell me why a creative was underperforming. It saw ROAS 0.9 and said "kill it." I needed to look at the actual ad to figure out if the copy was the problem, the thumbnail, the CTA, or the audience mismatch. The AI made the decision; I made the diagnosis.
Platform quirks: On Meta, it underweighted the "learning phase" window. A creative at 12 days with 20,000 impressions was still in Meta's learning phase and the model wanted to kill it. I overrode 3 of those.
Seasonality: It didn't know that my Q3 product launch was two weeks out and that certain audience segments would convert differently. I had to inject that context into the prompt manually.
None of these are AI failures. They're context failures. The model does what you tell it to do with the data you give it. The job is making sure the data is complete.
The Framework in Detail
Here's the scoring rubric I use, which is the actual "brain" of the system:
Efficiency (weight: 40%)
ROAS ≥ 2.5: 10 points
ROAS 2.0-2.49: 8 points
ROAS 1.5-1.99: 6 points
ROAS 1.0-1.49: 4 points
ROAS < 1.0: 2 points
Trend (weight: 30%)
7-day ROAS > 28-day ROAS: 10 points
7-day ROAS ≈ 28-day ROAS (±10%): 7 points
7-day ROAS < 28-day ROAS: 4 points
Exposure (weight: 15%)
50,000 impressions: 10 points
20,000-50,000: 7 points
15,000-19,999: 5 points
< 15,000: 3 points (flag as "observe, don't kill")
Substitution (weight: 15%)
No newer creative in same campaign: 8 points
Newer creative within 20% of performance: 6 points
Newer creative outperforming by 20-50%: 4 points
Newer creative outperforming by >50%: 2 points
Decisions:
Score ≥ 8.0: KEEP
Score 5.5-7.9: OBSERVE (review in 7 days)
Score < 5.5: KILL
This is not a neural network. This is a weighted scoring function that an LLM executes with more nuance and fewer edge-case bugs than I'd write in a spreadsheet. The LLM's value is in handling the qualitative edge cases: "this creative is for a product that just had a price change, so the ROAS dip in week 1 is expected" or "this is a brand awareness creative, so judge it on CTR, not CPA."
What This Looks Like Operationally
Monday 9am: Pull data from 4 platforms. 20 minutes.
Monday 9:20: Feed into prompt. 5 minutes.
Monday 9:25: Review 312-creative table, override 8-12 decisions. 15 minutes.
Monday 9:40: Execute kills and reallocations in platform managers. 30 minutes.
Monday 10:10: Done. 40 minutes total.
Versus the old Tuesday-morning ritual: 2-3 hours of dashboard-shuffling and gut-feel decisions, often with second-guessing that extended into Wednesday.
The Part I'm Not Telling You
The 47% jump didn't come from the AI being smarter than me. It came from the AI being consistent. I used to keep the creative I liked. The model keeps the creative that's working. It doesn't have a favorite. It doesn't get attached to the one with the cute thumbnail. It doesn't give the expensive creative one more day because "it's been working for three weeks."
The AI removed my sentiment from the kill list. And in performance marketing, sentiment is a tax you pay in CPA.
If you're managing more than 50 active creatives, you're making more keep/kill decisions than you can make well by eye. You don't need a PhD in ML to build a version of this. You need a scoring rubric, a data pipeline, and the discipline to let a system make the mechanical decisions so you can spend your judgment on the creative ones.
Kill the ads that deserve to die. Feed the budget to the ones that deserve it. The revenue does the rest.
