Hotspot vs tooltip for a new feature: +26% feature use
Commentary byLiftArchiveOur analysis, not theirsHotjar announced a new feature with a subtle hotspot instead of a tooltip and feature use rose from 6.5% to 8.16%, a 26% lift.
Hotjar ran the A/B test, and it was published as an Appcues customer story.
Users who saw the hotspot were also more likely to highlight recordings.
- Use of the relevance selector: 6.5% with the tooltip vs 8.16% with the hotspot, a 26% relative lift.
- Users highlighting part of a recording, a secondary metric: +99% with the hotspot.
- Sample size, test length and statistical confidence are not shared.
The change
The variant swaps the tooltip for a pulsing hotspot with the same copy.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
A tooltip announced the new recording relevance selector to a random half of users.
Variant
A hotspot, a small pulsing beacon, carried the same copy and was shown to the other half.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A hotspot invites users to click when they're ready, so it may feel less like an interruption than a tooltip.
- Choosing to open the message may make users pay more attention to it.
- More use of the relevance selector likely surfaced more useful recordings, which would explain more highlights.
Caveats: Published by Appcues, the tool vendor used for the test. Sample size, duration and confidence are not shared.
Who ran it
Hotjar ran this test and published the results (Appcues customer story). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Hotjar →
How to run this test on your site
- Hypothesis
- If we announce a new feature with a hotspot instead of a tooltip, then more users will try it because the prompt feels optional and less disruptive.
- Primary metric
- Share of exposed users who use the new feature
- Guardrails
- Repeat use of the feature after first try
- Dismiss rate of in-app messages
- Engagement with the core workflow
- Who should test it
- SaaS products launching features inside an existing screen. If the feature is critical to setup, a more direct prompt may still work better.
Questions
What is the difference between a hotspot and a tooltip?
A tooltip shows a message right away next to an element.
A hotspot is a small pulsing dot that shows the message only when the user clicks or hovers on it.
Are subtle in-app announcements better than modals?
Often for minor features, since they interrupt less.
In Hotjar's test the subtler hotspot beat the tooltip, but test it for your own product and feature.