By Ibby Rahmani
Published on August 31, 2021
In the previous blog, we discussed how Alation accelerates your journey to the Snowflake Data Cloud. Alation provides visibility, understanding, and compliance during the whole migration process.
In this blog, we will discuss how Alation provides a platform for data scientists and analysts to complete projects and analysis with speed.
How will you support your key users in the Data Cloud? Your data scientists and analysts will need access if they’re to conduct modeling and analysis at speed. Typically, these groups spend the bulk of their time searching for trusted data before they can actually use it.
So for the average enterprise data consumer, what’s the search to usage ratio? An IDC data culture survey of 455 data-native workers in North America asked how much time people spend on data activities, from searching to preparing to protecting to analyzing. The survey results suggest that this ratio is closer to 70/30.
It’s clear that data consumers waste many hours hunting for good data. They need a platform that makes it easy for them to find and understand data quickly in the Snowflake Data Cloud. The platform needs to support search & discovery that empowers understanding, spotlighting that which is most relevant to the searcher. Finally, it should make collaborative work around data seamless, providing a single source of reference for a range of users.
Data must be:
Understandable with context about past usage, popularity, and transformations over time
Discoverable with user-friendly search
Relevant to the person who is searching
The result is a collaborative platform, where people are empowered to learn from the successful projects of their peers, and work together, sharing queries and best practices.
Top-notch discoverability requires two elements: connectivity and access. A data catalog must be able to connect to all data sources – from legacy sources to the Snowflake Data Cloud – and deliver big-picture visibility of an organization’s entire data landscape. Second, searchers need access in order to capitalize on that visibility.
Data should be easy for users to discover. Data scientists and analysts need a Google-like search capability with a natural language interface. You should be empowered to search using their everyday language rather than coded terms.
Once you discover your data in the Snowflake Data Cloud, you need to understand if and how you can use that data. For that you need important information, which will guide your own usage.
Data scientists and analysts know that the information is available somewhere, but only a small number of senior employees will know where to look. In most cases, they have data stored in different systems and don’t have a centralized documentation of the data. Even phrasing the query properly is tricky: What search terms would you use to request, “that table that lists share events”?
To address this issue, organizations have resorted to in-house solutions. But they are not always scalable – especially for the Snowflake Data Cloud. The problem intensifies as the business grows, as assets proliferate and the disconnect between IT and data scientists & analysts widens. As the trend continues, understanding the data becomes more difficult. They need a single system of reference where they can have a common data dictionary, manage business glossary, and find subject matter experts.
Searching and discovering data is only half the battle. To make quick decisions backed by data, you need access to the right data. However, with more data than ever before, the ability to find the right data has become harder than ever. Data scientists and analysts are presented with an overwhelming amount of information, and just want visibility into the most relevant data.
Sometimes you just want data in a single category: one domain. Users don’t want to wade through all the data in the Snowflake Data Cloud. They need to be able to search data within a single department or category, such as marketing data or customer data in Finland. Domain labels empower users to quickly see what’s most relevant to their given task. It reduces the likelihood of newcomers feeling “inundated” by too much information, and makes the most relevant data the most visible. This accelerates adoption, and gives every user immediate, role-specific data.
Data users in organizations need to find data to make data-driven decisions. However, data scientists and analysts face the challenge of finding data easily. This challenge is further compounded by limited access to trusted data, often spread across disparate tools. They need a tool that helps them connect to all the data in the organization and leverage existing reports, dashboards, and visualization created by colleagues.
Thousands of hours are wasted on duplicated work. Data scientists and analysts are often unaware of existing reports, dashboards, and visualizations created by colleagues. They need a central place that connects everyone to share knowledge and experience. This can be accomplished through collaboration.
Collaboration is necessary for capturing tribal knowledge and making it available to the entire organization. This has become more important as remote work has become the norm. Without collaboration, data consumers (and their knowledge) are siloed, and work is needlessly recreated.
Enter Alation. Alation makes it easy for all users to find and understand the data in the Snowflake Data Cloud. Analysts can focus on creating dashboards and reports, data scientists can build models without friction, and teams can collaborate globally toward desired business outcomes.
Alation provides visibility into all the existing and Snowflake data sources. Alation connects to legacy and Snowflake data sources and surfaces intelligence. All users now have a single place to search for data, and are spared having to hunt through multiple dashboards.
By relying on clues in the data and natural language search, Alation helps you to find the best data. This makes the data cloud accessible to everyone. The result: They save precious time to do important things – modeling and analysis.
Alation translates technical terms to business language. Using behavioral intelligence, Alation interprets organizational usage to automate a business glossary. This glossary serves as a common vocabulary for the organization, helping everyone to use the right terms consistently.
Crowd-sourced descriptions, conversations, and wiki-like articles empower you to better understand every asset in Snowflake Data Cloud. This results in higher productivity and collaboration.
The Behavioral Analysis Engine (BAE) leverages machine learning with pattern recognition to surface insights on how data is being used in the Snowflake Data Cloud. Machine learning spotlights each asset’s classification, popularity scores, and top users.
Alation enables you to find the most relevant information faster with faceted search and data domains. Data domains reflect how an organization sorts data. For instance, users can create domains by “business area,” “use case,” “geography,” and more. Seekers can easily customize their search to home in on the right data for their question. In this way, users can quickly discover information that is most relevant to them.
Alation facilitates collaboration on the Snowflake Data Cloud, so that no data stakeholders work in isolation. The data catalog becomes the centerpiece connecting people, data, and use cases in a way that improves both speed and quality of analysis.
This is the human side of the Alation Data Catalog, which breaks down organizational silos and fosters a culture of sharing: knowledge sharing, data sharing, process sharing (data preparation), and analysis sharing.
The Alation Data Catalog empowers data scientists and analysts to access the Snowflake Data Cloud. Alation speeds their modeling and analysis by providing a unified platform to find and understand relevant data in the Snowflake Data Cloud. With effective collaboration, each contributor works toward a common goal, building off the work of others, and opening the door for greater innovation.
In the next blog, we will discuss how Alation helps you with Governance.