Intelligent Search¶
Alation Cloud Service Applies to Alation Cloud Service instances of Alation
Overview¶
Intelligent Search is Alation’s advanced search solution, blending Semantic Search (or AI Similarity Search) and Keyword Search to enhance the discovery of information in the data catalog. Intelligent Search is in use by default on Alation Cloud Service instances on the cloud-native architecture and both New User Experience and Classic User Experience.
Note
Intelligent Search is accessible on instances at the Deluxe tier and higher. Instances at the Essential tier use Keyword Search.
Intelligent Search uses a dual-search mechanism that processes search queries through both Keyword and Semantic Search methods:
Keyword Search functions by matching specific words or phrases within the content.
Semantic Search goes beyond keywords to understand the context and meanings of words in search queries. By understanding the context of search queries, Semantic Search can connect related concepts, such as, for example, linking Bay Area with San Francisco and California. Searching for Bay Area will also yield results about San Francisco and California because those words are connected to the context of Bay Area.
In Intelligent Search, the transition between Keyword and Semantic Search is seamless, requiring no additional actions from users. You enter search queries as usual, and the system automatically employs both Search technologies to deliver the most relevant results. The final search results represent a blend of both Semantic and Keyword Search outputs, with semantic output ranked higher. Semantic Search potentially reduces the need for additional filtering, as it presents the most relevant and precise results.
How It Works¶
Alation utilizes a pre-trained sentence transformer LLM (Large Language Model) to interpret and vectorize content as part of Semantic Search. This model converts the text into numerical vectors that capture the essence of the content. When a search query is entered, it undergoes the same transformation process to ensure that the meanings are represented as vectors. These query vectors are then compared to the content vectors stored in the system. The most relevant content is identified based on conceptual similarity in vector space. The search results returned are those whose vectors are closest in meaning to the query.
Content Privacy¶
Alation’s sentence transformer model is pre-trained with public data. Data from customer instances is not used for any model training.
Language Support¶
Alation’s Intelligent Search only supports English.
Get Intelligent Search Results via the Search API¶
Intelligent Search will work through the Search API without any change.
Disable Intelligent Search¶
Customers have the option to disable Intelligent Search if desired. Such requests are processed through a Support ticket. If disabled, the system will revert to using only the Keyword Search method.
Frequently Asked Questions¶
What are the benefits of Intelligent Search over Keyword Search?
Precision and Relevance—Concepts are more robust than keywords. By matching concepts rather than keywords, semantic search produces more relevant results.
Ease of use—If a search string is clear and the data catalog content is well curated, Intelligent Search is more likely to display the most relevant objects at the top of the search results. This enhanced accuracy can reduce the need for users to apply additional filters, simplifying data discovery.
Will Intelligent Search be available on customer-managed instances?
Since Intelligent Search requires an infrastructure expansion and index migration, at this point we have no plan to offer Semantic Search on customer-managed instances or Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.
Is Intelligent Search enabled by an underlying knowledge graph?
It’s not. Alation’s Intelligent Search utilizes a pre-trained sentence transformer LLM to extract the meanings of contents and represent them as vectors. No knowledge graphs are in use.