The $1 Million Lesson: Why "Best Practices" Keep Losing A/B Tests

You know the rules. Keep onboarding short. Show the right paywall at the right moment. Offer a free trial so people can try before they buy. Get featured by Apple and watch the downloads roll in. These ideas get repeated in every growth thread and design review, often enough that nobody questions them anymore.

Moonly questioned them, with real money on the line. The astrology app spent more than a million dollars on A/B tests over six years, grew to $20 million in revenue without investors, and now converts about 40% of users who install it from an ad. Along the way, nearly every piece of conventional advice failed its test. What worked instead is a useful look at how users actually behave.

The Myth of Short Onboarding

We treat every extra onboarding screen as a place where users drop off. But many top-grossing apps have some of the longest onboarding flows in their category, and Moonly's runs about 30 screens.

Length wasn't the problem. Focus was. Over three major redesigns, the team shifted from showing off features to addressing the user's actual problem. The current flow names the pain point first, then gives a small, genuinely useful insight, such as how lunar cycles might affect mood. Only after that does it introduce the feature that solves the problem. By the end, the product has already proven its value.

Then they went further. With 40 features and users who only care about a few, Moonly built its own attribution engine. It connects each ad to a custom App Store product page and then to a matching onboarding flow. Someone searching for tarot gets tarot screenshots and a tarot-focused experience. There are now around ten flows in production, each built around a specific user need, including anxiety and ADHD.

The result: customer lifetime value doubled. Ad spend doubled within a month, and cohorts became profitable from day zero.

Why Removing Things Beats Adding Them

Only about 12% of apps have no sign-up or login step. Moonly has 10 million users and no login screen at all.

It started with a simple question: why do we need this? Designers add login screens because every other app has one. So the team tested each justification. Restoring data across devices? Apple Keychain already does that. Collecting emails? Typing a real email adds friction, and most users hide theirs behind Sign in with Apple anyway. Email marketing? After years of effort, it was still expensive and hard to make work. Once no reason survived, the screen went away.

Contextual paywalls got the same treatment. Showing a tarot paywall on the tarot screen and a birth chart paywall on the birth chart screen sounds like textbook personalization. Every single contextual paywall lost its test. When users saw different paywalls for different features, they assumed each one had to be bought separately. A single, consistent paywall that makes clear one payment unlocks everything won.

The Paywall Lever Nobody Expects

After hundreds of experiments on every element of the paywall, the biggest lever turned out to be the image.

Within a few days, one winning image delivered a 2x uplift, and it wasn't the one the team had bet on. They had fallen for a Disney-like 3D style. The winner was a realistic photo of a woman at sunset. Moonly's most valuable audience is over 35, and those users couldn't see themselves in cartoon characters.

Realism has limits, though. The team once personalized around 50 in-app images with each user's own face. It was complex, expensive, and sometimes deeply awkward. Thousands of users started avoiding the app, and the feature was removed. People are very sensitive about anything involving their own face.

Smaller details added up too. Changing only the words in the title and subtitle raised conversion by 12%. Social proof, such as download counts and reviews, consistently helped, and so did a simple "no commitment, cancel anytime" line. Paid-traffic conversion rose from 10% in 2020 to 40% today, mostly through small improvements shipped almost every day.

The Psychology of Price

Moonly ran 40 pricing experiments in a single year, and the most useful lessons came from understanding how people compare prices.

Price anchoring — a high-priced lifetime plan makes the whole product feel more valuable and pushes users toward a recurring plan. The goal isn't to sell many lifetime plans. Only about 4% of iOS apps offer one, usually priced at roughly twice the annual plan.

Same number, different unit — people compare prices to each other, not to value. So Moonly kept the numbers and changed the billing period: $8 a month became $8 a week, and $30 a year became $30 a month. Weekly subscribers churned faster, but they paid more times before leaving, so lifetime value went up. Cash also came in sooner, which meant faster ad-spend recovery and a shorter feedback loop for scaling.

Killing the free trial — Moonly ran a trial for about a year, but unit economics were almost always stronger without one. Many users started a trial and cancelled within seconds. Worse, trial users got the birth chart for free, one of the most valuable features, and then felt they had already received most of the product's value. The trial was replaced with a one-time discount offer around day three for users held back by price. That single offer lifted revenue between day three and day thirty by about 15%.

What Actually Moved the Needle

Beyond the tests, four bets paid off in a big way.

One-tap sharing. People don't share features; they share insights about themselves. When a user takes a screenshot, Moonly offers a ready-made share flow. About 23% of users share something, generating millions of free views.

An AI astrologer that feels trustworthy. Luna reached about a million conversations in six months. Cartoon, Disney-style, and faceless versions all failed. When people ask for personal guidance, the guide has to feel credible, and realism won again.

Add-ons that don't cannibalize. Moonly built its own Vedic calculation engine and around ten in-depth reports on topics like soulmates, career, and numerology. These reports also feed into Luna's context. The add-ons are already on pace for around $500K a year, sold through a context-aware announcement system and through Luna's recommendations when a report would actually help.

Moving payments off Apple. With native iOS payments, half of all cancellations happened within three minutes of purchase. After Moonly added Stripe in the US with a small discount, LTV doubled overnight, and about 80% of US customers now pay that way. The same approach doesn't work well on Android. Google's alternative billing gives developers little control, doesn't support Google Pay, and kills conversion.

What This Means for Product Design

If you're designing an app's growth and monetization flow, Moonly's six years suggest a different starting point:

• Question every default. Before adding a login, a trial, or a contextual paywall, ask why it needs to exist. "Everyone does it" isn't a reason.

• Personalize the path, not just the pixels. Matching onboarding to the reason someone arrived did more than any visual redesign.

• Design for the audience that pays. The style your team loves may not be the one your paying users recognize themselves in.

• Let small wins compound. A 40% conversion rate came from years of daily increments, not a single redesign.

• Stay lean before you scale. Urban's advice for new builders is to start with a web funnel. It gives you more freedom and faster market signals, and one person with AI agents can build it.

At Moonly's pace, this only works with systems. The team uses Cursor and Codex to design experiments and pull data from sources like PostHog and Adapty, then gets clear recommendations: ship, don't ship, or collect more data.

The Honest Verdict

Best practices are useful as a starting point. But they're usually averages drawn from other people's products, audiences, and contexts. What worked for a meditation app or a fitness tracker may quietly fail for yours.

Moonly's biggest advantage wasn't any single insight. It was the willingness to test the things nobody questions, until every icon, line of text, and screen had earned its place.

Maybe the most important question isn't what's the best practice?

Maybe it's have we actually tested it?

This article was inspired by the video "He Spent $1M on A/B Tests. Here's What Won." by Mobbin, featuring Moonly founder Vitaliy Urban. Watch it here: https://www.youtube.com/watch?v=qK7WYCMvjUw