Switching site search to Google Vertex AI: +20.6% conversion rate
Commentary byLiftArchiveOur analysis, not theirsReplacing the Boost Commerce search app with Nimstrata's Google Vertex AI search on search and collection pages raised conversion rate by 20.6%.
The Shopify agency 36 ran this test with Nimstrata for The Bee's Knees British Imports, and it was published as a Shoplift success story.
The test ran for five weeks and reached 99% confidence.
- Conversion rate: +20.6%, reported at 99% confidence.
- The case study says the AI's results improved over the test as it learned from shopper behavior.
- No revenue or order value figures were shared.
The change
The variant swaps the search engine and filters behind the same pages.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
Search results and collection pages powered by the existing Boost Commerce search and filter app.
Variant
The same pages powered by Nimstrata's Retail Cloud Connect app, which uses Google Vertex AI Search for Commerce and filters that adapt to each query.
Why it likely won
Our reading of the result, not the practitioner's claim.
- The old search often returned 'No Results Found', which sends shoppers away; better matching keeps them browsing.
- Filters that change with the query show only useful options, so shoppers reach the right product faster.
- A grocery import store has many niche items with odd names, where smarter search helps most.
Caveats: This is a comparison of two vendor apps, reported by the vendor side. It tests the whole search engine, not one design element.
Who ran it
36 Agency ran this test for The Bee's Knees British Imports and published the results (Shoplift success story). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on 36 Agency →
How to run this test on your store
- Hypothesis
- If we replace our current site search with a search engine that matches intent better, then conversion rate will rise because fewer searches end with no results or the wrong products.
- Primary metric
- Conversion rate
- Guardrails
- Searches ending with no results
- Revenue per visitor
- Page load time on search and collection pages
- Who should test it
- Stores with large or varied catalogs where many visitors use search or filters. It probably won't matter much for small catalogs where most shoppers buy from one or two product pages.
Questions
Does better site search increase conversion rate?
It can. In this test a new AI search engine on search and collection pages lifted conversion rate by 20.6% over five weeks.
How do you A/B test a search app on Shopify?
This store used theme testing to show half of visitors the old app and half the new one on the same pages, split evenly across desktop and mobile.
What are dynamic filters?
Filters that change based on what the shopper searched, so only relevant options appear and fewer clicks lead to empty results.
Similar tests
@oliverkenyon@oliverkenyon
ConvertiblesConvertiblesConvertiblesConvertibles