Rebuilt onboarding tooltips plus test series: +10% lesson opens
Commentary byLiftArchiveOur analysis, not theirsLingQ rebuilt its home-made onboarding tooltips in Appcues and then ran a series of messaging and tour tests, raising its lesson open rate by 10% overall.
LingQ's learner success team ran the work, and it was published as an Appcues customer story.
The 10% is the combined effect of the rebuild and many tests, and the team says most of the individual tests did not win.
- Lesson open rate: up 10% overall, as reported by Appcues.
- The rebuild alone raised the onboarding funnel, but no figure was given for that step.
- The team says most individual tests in the series lost; the gain came from keeping the few winners.
- No sample sizes, durations or per-test results were shared.
Control
New users of the LingQ language-learning web app saw onboarding tooltips that the team had built in-house.
Variant
The team first recreated the same tooltips in Appcues, which alone lifted the onboarding funnel. They then tested different tooltip messages, experience types and product tour variations with split groups of new users.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A tool the team could change without developers made it practical to run many small tests.
- Testing each point of the funnel let small wins add up.
- Keeping only the winners means a mostly losing test series can still produce a net gain.
Caveats: Bundled: the 10% combines the rebuild and a series of tests, so no single change earns the credit. Vendor-published, with no sample sizes or per-test detail.
Who ran it
LingQ ran this test and published the results (Appcues customer story). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on LingQ →
How to run this test on your site
- Hypothesis
- If we test tooltip messages and tour variants at each step of onboarding and keep only the winners, then more new users will reach the first core action because each step gets clearer.
- Primary metric
- Share of new users who open their first lesson (or your first core action)
- Guardrails
- Onboarding completion rate
- Week-one retention
- Tooltip dismissal rate
- Who should test it
- Consumer or prosumer apps with enough new sign-ups to run tests back to back. With low sign-up volume, each test will take too long to read.
Questions
Is it normal for most onboarding tests to lose?
Yes. Many teams find only a minority of tests win.
LingQ's team said the same, and still gained overall by shipping the winners.
Should I build onboarding tooltips in-house or use a tool?
In-house works if changes are rare. A dedicated tool makes sense when you want to test and change onboarding often without waiting on developers.
What should I test in onboarding tooltips?
Start with the message on each step, then the type of experience, such as a tooltip versus a short tour.
Measure the action the step is meant to drive.