Detection: Top ad got more impressions than it earned

Key: false_winner_meta_bias Severity: High Confidence: 75–90%

What this detection looks for

We flag an ad set when its apparent "winning" ad got disproportionate delivery without correspondingly better conversion economics. The signal that Meta's delivery system is rewarding an early lead instead of the genuinely best-performing creative.

All of the following must be true:

1. The ad set is active
2. It contains at least 3 active ads
3. Every ad has run for at least 7 days (so all of them have had a real chance to compete in the auction)
4. The top-impression ad has at least 2× the impressions of the second-place ad in the same ad set
5. The top-impression ad's conversion rate (purchases ÷ clicks) is at least 15% lower than the combined conversion rate of the other ads in the ad set

Why this matters

Meta's delivery model decides which ad in an ad set to serve based on a combination of predicted action rate and historical performance. The system has a well-documented tendency to lock in on the early leader: the first ad to get a meaningful conversion is rewarded with more impressions, which produces more conversions, which produces more impressions — a feedback loop that can persist long after a better sibling has entered the auction.

The result is a false winner. The ad with the highest impression volume is presented to the practitioner as the obvious top performer. The practitioner pauses the others to "concentrate spend on what's working." Account performance then degrades because the actually-best ad was the one that got starved out, not the one that won the lottery on day three.

This is the detection we polish hardest. If a user only ever runs Honest Growth once, this is the finding we want them to leave with.

How we calculate confidence

How we calculate the estimated monthly cost

We surface the dollar value of the lost conversion lift, projected to a 30-day month, as the share of top-ad spend whose lower conversion rate represents conversions that would have happened on a better-performing sibling.

monthly_cost = (top_ad_spend × cr_deficit) × (30 / days_in_range)

For example: if the top ad has spent $3,000 in the date range and its conversion rate is 20% lower than the rest, we surface $3,000 × 0.20 × (30/30) = $600 as recoverable monthly value.

This is the underperformance of the impression-dominant ad, not its total spend. The fix is to redistribute future impressions, not to turn the top ad off completely.

What would change our mind

This finding can be a false positive in a small number of cases:

• The top ad is genuinely the best. Sometimes the early leader is also the best long-term performer. If the top ad's CR is similar to or better than the rest, we do not surface a finding. The 15% deficit threshold is what separates "winning fairly" from "winning by attrition."
• The other ads have only recently launched. A new ad that has been live for 5 days will have low impressions because Meta is still exploring it. We require every ad in the comparison set to have at least 7 days of data to avoid this case.
• The CR difference is by design. Some teams intentionally include a retargeting-style ad in a prospecting ad set that has lower CR but higher brand value. If a specific ad is meant to underperform, the finding does not apply — but in that case the ad probably should not be in the same ad set as the prospecting creative.

How to fix it

1. Do not pause the top ad. That is the natural instinct and almost always wrong. The top ad is producing conversions and pausing it removes a source of qualifying events.
2. Duplicate the ad set with all the same ads. In the duplicate, manually allocate budget across ads (manual delivery) or use Meta's Advantage+ creative randomization. This creates a fresh auction where the historical-winner bias is reset.
3. Run the duplicate alongside the original for 7–14 days at equivalent budget. Compare the per-ad CR in both ad sets.
4. If the underperforming ads from the original ad set show meaningfully higher CR in the duplicate, you have confirmed the false-winner bias. Shift spend to the duplicate and retire the original.
5. If CR is similar across both ad sets, the original was a real winner. Keep the original; the finding was informational.
6. Going forward, when launching new creative into a mature ad set, launch into a duplicate so the new creative competes from day one rather than against a baked-in incumbent.

What we look at to make this detection

• effective_status on the ad set and each ad
• Per-ad impressions, clicks, and purchase conversions, summed over the audit date range from the Insights API
• Per-ad days with non-zero impressions to confirm each ad has had enough exposure to compete
• We deliberately do not use ROAS or revenue here, since revenue is noisier per ad than purchases-per-click and biases this detection toward higher-AOV winners. Conversion rate (purchases ÷ clicks) is the cleanest comparison

Source

This methodology page is generated from apps/api/app/services/detections/false_winner_meta_bias.py. The detection code is open for inspection. We do not have hidden rules.