How to Leverage an AI Shopping Assistant with RAG and Hypothetical Document Embeddings (HyDE)

Nektarios Sylligardakis -

Online retailers are always looking for ways to improve customer experience, and one of the ways they're doing this is by using AI shopping assistants. However, with so much information to sift through, these assistants can struggle to provide accurate and personalized recommendations using RAG. That's where Hypothetical Document Embeddings (HyDE) come in. By leveraging these technologies, retailers can improve the performance of their shopping assistants and provide customers with a more efficient and personalized shopping experience.

At helvia.ai, we recently engaged in a comprehensive discussion about the potential functionalities of a GenAI shopping assistant, focusing on how it could accurately identify the products a user desires or needs for specific tasks. This discussion provided the perfect setting to initiate an exercise I call "Product the Gap", which was designed to explore consumer needs ("Jobs") and possible solutions using the Jobs To Be Done (JTBD) framework. The insights gathered were visually represented on a Miro board.

In this blog post, we'll go through our approach using some real-life examples.

How to Leverage an AI Shopping Assistant with RAG and HyDE?

The Jobs To Be Done (JTBD) framework

The process began by addressing the primary question: What is the main job of someone looking to purchase a product online? From this, we formulated the core job statement (Main JTBD (1.0)): "Enable me to efficiently discover and select products that align perfectly with my specific goals and tasks, minimizing both effort and uncertainty throughout the decision-making process." This can be viewed on the Miro board, marked with a red sticker No. 1.0

To better understand the job context, we documented several job examples on the Miro board, each tagged with a red sticker (No. 2.x). Here, we will discuss The Bluetooth Example (2.2), which closely aligns with our solution:

"I want to buy a Bluetooth speaker for under $100 to use at pool parties."

Exploring how a user might currently find and purchase the best Bluetooth speaker under $100 for pool parties involves the following steps:

  • Search Google for articles reviewing Bluetooth speakers.
  • Identify waterproof models.
  • Conduct a feature and price comparison.
  • Decide on one.
  • Research where it can be bought at the lowest price.
  • Make the purchase.
  • Alternatively, the user might:
  • Ask friends for recommendations.
  • Search for model reviews on Google.
  • Compare prices.
  • Complete the purchase.

Simplifying the JTBD process with AI

Our solution simplifies this process: by requesting "three Bluetooth speakers for use at pool parties under $100," our AI will offer three options from our e-shop, complete with brief descriptions and pricing. You can then select one, add it directly to the shopping cart, or visit the product page. The response from this request can be seen under sticker 2.2.2 on our board at perlexity.ai.

Identifying the Product Gap (2.2.3)
We compared our AI solution with traditional methods to understand where our approach improves functionality:

(+) Reduces time spent researching.
(+) Quicker in finding and selecting products.
(-) Uncertainty about whether the best option was chosen.
(-) Traditional preference for Google searches over querying bots.
(-) Higher trust in personal recommendations over bots or Google.
(+) More suitable for advanced users familiar with AI, like ChatGPT.

This analysis raised an important question: "How can we integrate a search form as the start of a chat conversation?" We plan to explore this further in our design phase.

Repeating the Exercise
It is beneficial to repeat this exercise with different examples in varied contexts to fully understand the constraints and potential of our solution across different scenarios.

How It Could Work (3.1)
Our solution relies on an AI Assistant metaphor that is able to search over the product catalogue in a more effective way, typically through a Retrieval Augmented Generation (RAG) pipeline. However, creating an appropriate knowledge base for RAG using e-shop product data presents a challenge: the product descriptions may not explicit mention references to the product's suitability for specific uses like pool parties..

One idea is to create Hypothetical Documents using a language model to generate additional information about products, such as writing product reviews, or describing ideal usage scenarios or even describe the image of the product . This enriched content could significantly enhance the knowledge base, enabling it to accurately fulfill queries like "suggest three Bluetooth speakers for use at pool parties under $100." This is actually supported by recent research on what is called "Hypothetical documents Embeddings" which shows improvements in similar scenarios.

What are Hypothetical Document Embeddings (HyDE)?

Let’s imagine an e-shop planning to introduce a new Bluetooth speaker specifically designed for pool parties. This speaker boasts features such as high water resistance, extended battery life, and powerful bass suitable for outdoor environments.

To aid the e-shop’s recommendation engine in accurately categorizing and suggesting this new product, hypothetical document embeddings can be created. The system would generate an imaginary product description incorporating key features of both pool party-friendly equipment and robust Bluetooth speakers. For example, it might include elements from existing durable, waterproof speakers that are known for high sound quality outdoors.

This hypothetical description helps the recommendation system understand how this new speaker aligns with potential customer interests who often search for pool party gear or outdoor sound systems. By training on this constructed scenario, the system can more effectively suggest the speaker to those looking to enhance their pool party experiences, ensuring targeted recommendations even before actual customer feedback is gathered.

Testing Hypothetical Documents

To evaluate the practical application of hypothetical documents, we used them with three real products:

Test 1: IKEA HEJNE 3 Shelves (3.2.1)

We generated the following hypothetical documents using ChatGPT-4 with these prompts:

  • “Write a review for IKEA HEJNE 3 shelves”
  • “Best use of HEJNE 3 shelves”

These hypothetical documents could facilitate queries such as "I want shelves for my garage" helping to efficiently retrieve the related product linked to the hypothetical document, which would not be possible to retrieve using just a RAG.

View the Hypothetical Documents on the Miro board, marked with a red sticker (No. 3.2.1).

Test2: Amazon T-Shirt (3.2.2)

We generated the following hypothetical documents using ChatGPT-4 with these prompts:

  • “You are a stylist, can you describe the tshirt, what style it is and what other clothes I can match it with?”
  • “You are a stylist, can you describe in the image what she is wearing, what style it is and what other clothes I can match it with?”

The hypothetical documents supported queries such as "I want a retro-style T-shirt to wear with a denim jacket."

Test3: Vacuum Cleaner (3.2.3)

We generated the following hypothetical documents using ChatGPT-4 with these prompts:

  • “Can you write a review for Dreametech T30 Cordless Vacuum Cleaner, 90mins Long Runtime Stick Vacuum, 190 AW Robust Suction Handheld Vacuum, Cordless Vacuum with HEPA Filters for Hard Floor Stairs”
  • “For what kind of use Dreametech T30 Cordless Vacuum Cleaner is best for?

The hypothetical documents assisted with queries like "I want a vacuum suitable for pet owners that is easy to store."

These examples showcase the potential of leveraging Retrieval Augmented Generation (RAG) systems with hypothetical documents to effectively bridge the gap between complex consumer queries and relevant product suggestions.

Google it vs Chat it 

During our exploration, two other significant insights emerged:

  1. They typically think about which keywords will yield the best results for their desired outcome, rather than simply asking for what they want.
  2. Users have been conditioned to use search engines rather than chatbots. 

How can we determine the specific purpose a user has in mind when searching for a product?

Building on the second point, it might be interesting to reverse the Jobs to be Done approach. For instance, when someone uses a search field, we could prompt the language model to identify potential jobs that need to be done. For example, if someone searches for “Household Vacuum,” we could use the LLM to ask, “Give me only the Category/Context of the top 10 JTBD for users looking to buy a vacuum, based on popularity.” Tagged with a red sticker (No. 4.1)

The response of ChatGPT4:

Certainly! Here are the Category/Contexts for 10 different JTBD scenarios for users looking to buy a vacuum:

Efficient Household Maintenance
Allergy and Health Management
Family and Child-Friendly Homekeeping
Flooring Versatility and Care
Pet Ownership and Care
Professional and Commercial Cleanliness
Space Optimization for Compact Living
Accessibility and Ease of Use
Sustainable and Eco-Conscious Living
Vehicle and Furniture Maintenance

Now, we can use this list to ask the user “What is the use you want the vacuum for?“ and then use the Hypothetical documents to provide product results.

How might we integrate the user search flow with AI-driven conversational flows?

Implementing a prompt beneath the search box that asks, “What is the use you want the vacuum for?” could serve as a method to integrate the user search flow with AI conversational flows.

However, I am confident that there is ample room to explore more effective solutions.

Taking Your Shopping Assistant to the Next Level with RAG and HyDE

As we wrap up this exploration into the evolving role of AI in shaping the future of online shopping, it’s clear that hypothetical documents and advanced retrieval systems like RAG can significantly enhance the way we search and discover products online. Our experiments with the "Product the Gap" initiative and hypothetical documents have opened up new pathways to connect complex consumer needs with precise product suggestions.

If you are interested in applying this AI-driven approach to enhance your own products, we would love to collaborate. Contact us to explore how these innovative technologies can be tailored to fit your business needs and elevate your customer experience.

Together, let’s continue to push the boundaries of what GenAI can achieve in the world of online shopping, making it more intuitive, efficient, and aligned with user expectations.