How To Use Machine Learning to Identify Search Intent

Do you want to increase CTR and reduce customer acquisition costs? Learn how to use machine learning to identify search intent in this post.

Identify Search Intent

Business landing vector created by pikisuperstar – www.freepik.com

 

Far too many ad campaigns use keywords without having identified search intent first. Unfortunately this may exclude your customer entirely or at the very least put them much farther up the purchase funnel.

Search engine algorithms were once built around simple keyword searches. That’s no longer the case. AI and machine learning are revolutionizing intelligent search and ensuring that SERPs now provide smart results that satisfy user intent. 

Machine learning presents numerous business opportunities for SEO specialists and digital content marketers. These include providing refined niche content, reducing customer acquisition costs, improving leads and lead times, increasing click-through rates, and boosting the user experience overall.

In this post we’re going to look at how to identify search intent in order to target customers much more likely to buy.

How Does User Intent Help Your Business

There has been a major move from typical search engine optimization built around simple keyword searches to semantic SEO built around topics and contextual issues. 

Understanding user search intent means that a marketer can decipher what potential customers want and where they are in the marketing funnel. Thus, you can generate relevant content to fit the person at each stage of their buying journey.

Some of the advantages of understanding user search intent include:

  • Reducing negative or zero results from your content marketing strategy.
  • Relevance filtering, i.e., improving matching and results.
  • Improving CTRs.
  • Improving leads, reducing lead times, and increasing the time spent on your webpage which increases your EAT.
  • Reducing customer acquisition costs and optimizing marketing resources.
  • Providing an overall excellent user experience and tailoring your content type.

Your content strategy should always include a means to rank highly on the SERPs, and this should include satisfying user intent. Leveraging the full power of AI on search results means that you need to go a little deeper in understanding and categorizing user intent before you ever get to ranking. This is further elaborated in the next section. 

How Is User Intent Defined by Humans V/S AI?

As per the traditional marketing funnel or Hubspot’s sales flywheel, user intent can be placed into the following categories:

  • Informational queries – for example, “are there customizable dog kernels in California”? These are usually defined by the presence of words such as “how,” “what,” etc.
  • Navigational searches – for example, “where to find customized dog kernels in California.” Defined by the presence of hyperlinks, maps, etc.
  • Considerations/commercial intent – for example, “customized dog kernel reviews in California,” defined by keywords like “reviews,” “comparison,” etc.
  • Transactional queries – for example, “customized dog kernels under $1000 in California”, defined by such keywords as “buy,” “price,” “review,” etc.

“Dog kernels California” is pretty straightforward for a human subject to understand. But how does an AI such as Google’s RankBrain see the same? 

For Google’s search algorithm, a query of “dog kernels California” pops up transactional intent (price), navigational intent (map links), and informational intent (blog pages). Google does an excellent job of providing the most relevant results and removing ambiguities from the SERPs.

How to Identify Dual User Intent

User intent might not exactly mean the same as a search query, although we think of them synonymously. For a person, identifying different types of search intent is easy and almost intuitive based on the keywords used. Take a look at the image below:

keywords

Source: (Dumitriu & Popescu, 2020)

The results are from a sample keyword planner tool. Generally, humans can tell that “online women footwear” and “online women shoes” refer to the same thing. Or that “kernels” and “kernels for dogs” are the same thing. For a search engine employing AI/ML, the above keyword clusters around women’s footwear might pertain to much more and generate a richer set of results. 

From an AI/ML perspective, two methods of identifying dual user intent can be proposed. These include Surface Query Similarity (from stemming, word order, compounds) and Post-Search Behavior (clicks, conversions, etc.) that show similar results.

Surface Query Similarity may be used to represent how far related certain words or terms are to each other. For a digital marketer, this means understanding when a query is transactional, informational, navigational, or otherwise and how different transactional intents may be satisfied in one query.

In Post-Search Behavior, user-generated signals after a query has been entered can further be used to gauge search intent. Such signals include post-search clicks, reviews, and social media –  likes, comments, and shares.

How Can You Use Machine Learning for User Intent

For a digital marketer, machine learning tools may help understand user intent and predict user behavior by correlating other users’ characteristics.

If you’re just trying to do lightweight keyword research for a blog with no product to sell, then simple keyword research tools may be a better alternative to complex machine learning tools. However, if you are a startup e-commerce store trying to sell multiple products and achieve high conversion rates, AI analysis tools might be for you.

The obvious place to start in your SEO strategy would be to determine your buyers’ intent. Let’s say your e-commerce store specializes in women’s clothing and footwear, including some unique leather items and slippers. Let’s further assume that your e-commerce store targets customers in a general city metropolis, for example, Boston.

With this information, you can define your buyer intent and start keyword research. Keyword research starts with tools such as SEMrush, Google Keywords Planner, and Ahrefs.

Once you have your target keyword cluster containing terms such as “online women’s leather shoes” and “online women’s shoes in Boston,” you can perform more in-depth research of your keywords. Ideally, you should get a sizeable keyword cluster with several hundred or thousands of keywords that you can narrow down to the best results.

keyword magic tool

Source

 

Once you have your list of keywords representing queries, you can run this through a vectorization model such as TF-IDF, BERT, or FastText, as discussed in the next section.

1. Consider the Importance of TF-IDF

Unfortunately, machines cannot process raw text like humans. It’s necessary to convert the text into a format that is well understood by the machine (think of Natural Language Processing). That is also called vectorization of text, depending on the method used. 

TF-IDF and Bag-of-Words (BoW) are two techniques used to vectorize and, in effect, visualize data.

TF-IDF refers to Term Frequency – Inverse Distance Frequency and is a weighting method or frequency count for word embedding. TF-IDF measures how important a term is in a document or in a collection of documents called a corpus. TF-IDF doesn’t analyze semantic meanings and is therefore limited in some regard, although it is computationally inexpensive compared to other models such as BERT.

Source

Besides using Bag-of-Words and TF-IDF, other methods of vectorization include:

  • Word2Vec – an algorithm that uses 2-layered neural networks to produce a set of vectors from a corpus. Unlike TF-IDF, Word2Vec takes into consideration the word context.
  • BERT – refers to Bidirectional Encoder Representations from Transformers and is an ML/NLP technique developed by Google to vectorize phrases, words, lemmas, stems, etc. BERT uses deep neural networks compared to TF-IDF and is, therefore, more computationally expensive.

Source

Choosing an MLOps company with a vectorization program to run your keywords is an integral part of the machine learning process. That’s where raw data (in effect, search queries) are labeled as informational, navigational, transactional, etc. 

2. Use SEO Tools to Define Article Outline

Once you have clearly defined your keyword intent and generated your short-tail and long-tail keywords from a model such as BERT, it is time to refine your content strategy. An AI writing tool can help you achieve this by improving your writing and editorial style, enhancing collaboration, and using SEO best practices to create high-quality content.

Source 

In the case of an AI tool, as captured above, teams can create clear, impactful copy while juggling multiple brand voices. Useful AI tools can assist with grammar correction, plagiarism checks, and suggestions on tone, spelling, wordiness, etc. 

You can create even more in-depth article outlines using applications such as SurferSEO and Frase. These provide measures to tag your article with H1, H2, and H3-H6, and you can then optimize your article further to yield better results. 

3. Use SEO Tools for Keyword Optimization

Even with output from tools like Writer.com or SurferSEO, you still need to optimize your content further. That ensures that your content, such as pillar pages, landing pages, product pages, or even blog articles, are competitive and fulfills search intent, especially with regard to keywords.

From the title, descriptions, headings, and sub-sections, each part of your document needs to be optimized. Remember, AI cannot perfect the work of optimization for you. Therefore, you need to polish your articles with a final human touch. That includes ensuring there is sufficient keyword density in your content for the target keywords you want to rank for.

For example, for product descriptions, you could provide rich snippets using schema data and add details such as pricing and reviews, which will make your pages rank higher in the SERPs. 

In Closing

Understanding and identifying user search intent is key to your SEO strategy. As a business, you can push relevant content, improve organic search results, reduce bounce rates, improve conversion rates, and reduce lead acquisition costs by coupling AI/ML with SEO.

AI/ML is not meant to replace human skill and intuition for SEO. However, combining your content strategies, such as keyword research, with AI/ML introduces scalability and refinement, allowing you to perform analysis across entire niches running into thousands, maybe millions of keywords.

Adding techniques such as keyword vectorization and schema optimization produces even better results. SEO specialists, content marketers, and other e-commerce experts thus have a powerful tool in their hands for advancing and scaling their content marketing strategies.