I was checking out some patents from Google the other day and surfaced an interesting one that was granted and published on May 30, 2023. It was titled “Generating Query Variants Using A Trained Generative Model” and it definitely piqued my curiosity. It was originally filed in 2018, but was granted in late May. And since I’m always interested in important SERP features like People Also Ask (PAA) and People Also Search For (PASF), I had to dig in.

Also, this is something I would have pinged the brilliant Bill Slawski about in the past. Unfortunately, Bill is not with us anymore. Going through the patent made me realize how much I miss Bill’s posts about patents and being able to DM him questions about his analysis. Losing Bill was definitely a massive loss for our industry. Anyway, without Bill to dig in the way he always would, I decided to start digging in myself. And I’m glad I did. It was super-interesting.

Leveraging Generative Models Using Neural Networks For SERP Features
Below I’ll cover how the patent describes using a trained generative model to generate query variants for SERP features like “People Also Search For”, “People Also Ask”, and maybe more. The patent mentions “People Also Search For”, but it’s not a stretch to believe the process could be used for PAA as well. I cover that in my analysis below.

It was fascinating to learn more about what Google is doing on this front (at least based on the patent). Like with any patent, we don’t know if Google has implemented this yet, or if they will, but it sure made sense based on what I was reading.

In addition, and I found this fascinating, the patent explained how Google could even generate query variants for novel queries (brand new), and long-tail queries where there isn’t much data available yet. And with 15% of all queries never seen by Google before, it would make sense to use an approach like for generating query variants. I’ll cover more about this soon.

Key points from the patent:
I think the best way to cover the patent is to bullet out some of the highlights. Below, I’ll cover several key points from the patent, which I hope you find interesting as well.

Generating Query Variants Using A Trained Generative Model
US 11663201 B2
Date Granted: May 30, 2023
Date Filed: April 27, 2018
Assignee Name: Google LLC

1. Query variants can be generated at run-time utilizing a trained generative model based on tokens from the original queries and additional input features. I’ll cover more about the additional input features soon.

2. The system can generate query variants even when the model is not trained on that query. So it can generate variants for novel queries (brand new) or what Google calls “tail” queries where there isn’t a lot of data yet. I found that very interesting, especially since Google says 15% of queries have never been seen before. So the generative model can predict which query variants to generate even for low-threshold queries by using a neural network (with memory layers).

3. The generative model can be trained based on submissions of previous queries by users. But the patent also explains that the query variant training data can also be based on query pairs that have clicks on the same documents. That makes sense and shows how user engagement can play a factor in what is generated by the model.

4. The patent also explains that the model can be trained as a multitask model to enable the generation of multiple types of query variants. So it’s a sophisticated system that can generate different types of query variants, including follow-up queries, generalization queries, canonicalization queries, language translation queries, entailment queries, and more.

5. After the query variants are generated, they are scored by the model. The system provides response scores for each variant. And the system can grade those variants by checking for answers to those query variants. That can help the system detect “potentially fake” query variants. Very interesting…

6. The patent goes on to explain that the system can return answers in addition to just query variants. For example, the system can return a search result (PAA anyone?), a knowledge graph entity, a null response (no answer), or even a prompt for clarification (with clarifying user interface input). That could be in the form of disambiguation chips we see when Google is looking for help from users when trying to understand what the user is looking for. Again, interesting.

7. The patent goes on to explain that the model can take more than just tokens from the query, including “additional input features”. Those input features could include location, a task the user is interested in or performing (like cooking, repairing a car, travel planning, etc.). It can also take into account weather and more. And the task could be based on stored calendar entries for the user, chat messages or other communications, past queries submitted by the user, etc. So the query variants could be based on personalization or current context.

8. The model can also generate variants of a query and advertisements or other content. So the model can not only generate query variants, but it can generate (or maybe retrieve) ads or other content that can be displayed in the SERPs. I think I have to go back through that section again, but that was interesting… 🙂

9. The patent also explains that there can be a number of generative models based on different attributes or tasks. So there can be specific models for various tasks like shopping, traveling to a location, etc.

Summary: Generating variants for PASF and PAA can be more complicated and nuanced than some think.
I hope breaking down this patent a bit helped you understand how Google could use a trained generative model to generate query variants, or other content, that can be displayed in various SERP features. And this can happen for novel queries (new) and long-tail queries where there isn’t much data yet. In addition, there could be multiple models being used that focus on a specific discipline. And the results can be personalized as well (based on additional input features).

So, the next time you view “People Also Search For” or “People Also Ask” in the SERPs, know that a generative model might have been used to provide those query variants. And if personalized, then maybe those queries are specific to your case. Again, Google’s systems are much more sophisticated than some people think.

GG