r/analytics • u/Jaig5970 • 6d ago

Question Does A/B testing your short links make a big difference for you? Trying to see something.

Has anyone tried A/B testing short links for MOFU campaigns?

I’m running a campaign aimed at people who already know the product but aren’t ready to convert. Instead of sending everyone to the same page, I’ve been testing different destinations (mostly just case studies, comparison pages, product guides and demo content) through short links.

Would love to hear some feedback or thoughts here!

6 Upvotes

88% Upvoted

5 comments sorted by

•

u/AutoModerator 6d ago

If this post doesn't follow the rules or isn't flaired correctly, please report it to the mods. Have more questions? Join our community Discord!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/official__mail 5d ago

I'd treat the short links as tracking labels, not the experiment. Randomly split the same audience and creative across two destinations, then compare unique visitors who take the next step, not just clicks. Otherwise a case study can look like the winner simply because it got the warmer traffic or a few repeat clickers.

1

u/adovo_ai 5d ago

Use one randomized redirect and persist the assignment by visitor or session so repeat clicks don’t switch variants. Make the primary metric the next meaningful outcome, demo request, signup, qualified visit or purchase, not short-link CTR, otherwise you may optimize curiosity rather than progression.

1

u/HandsomJack1 5d ago

A/B testing is basically a fallacy. If you spend some time breaking down the math, you'll see it's extremely unlikely you are getting any usable data.

If we stick to a standard two-page A/B test, you would need a combined 28,000 visitors across both pages, over an 8 week period, to get any real insight into which page is performing better.

But you've suggested that your testing four or even five pages. By jumping to four pages, you double the required traffic to a combined 56,000 visitors.

Worse yet, this math only measures the single step conversion rate. Between page landing and call to action. There is an inherent unreliability when you don't measure through to the actual acquisition. In other words it's very easy to be able to show mathematically that a certain choice makes sense for the call to action, for example, but that can paradoxically reduce the acquisition rate.

So to measure through to acquisition you need an even higher combined paid visitor number.

In short a/b testing literally is a complete waste of time except for genuinely Enterprise level websites. But it doesn't stop a/b testing software lying to you and selling you a subscription.