The Tribal Knowledge Problem

The second Alation State of Data Culture Report released today. The latest report shows that enterprises continue to make progress toward creating a data culture even while the pandemic has forced them to make tough decisions. Nearly all respondents report having lost personnel due to economic concerns.

But, things are looking up. More than half of data professionals believe their companies will be hiring in 2021 and nearly three-fourths say it’s likely that their budget will be restored. Unfortunately, even as personnel and budgets return, replacing lost tribal knowledge is more challenging — if not impossible.

The Tribal Knowledge Problem

If you’ve ever asked about a specific data set and been told, “Try asking Lucy in accounting,” you understand what tribal knowledge is. Tribal knowledge refers to the knowledge in people’s heads. Without a place to easily capture tribal knowledge, individuals tend to keep their knowledge to themselves — creating silos that hinder productivity.

Under normal circumstances, reliance on tribal knowledge is a hindrance. If someone has a question about a data set, they must either ask around and hope to find someone who can answer their question (a process made even more difficult with the move to remote work) or needlessly recreate work that has already been done. But during the pandemic, the over-reliance on tribal knowledge has become a major problem.

Almost all companies (96%) say that they have lost critical tribal knowledge from staffing changes during the pandemic. Training or retraining employees on how to use data has risen to the top of the list of initiatives for fostering a data-driven culture — up 37% from last quarter. And while training can make up for some of the lost tribal knowledge, much of that knowledge will simply be lost, putting a huge burden on data & analytics teams to start from scratch.

Making Tribal Knowledge Shared Knowledge

The pandemic has shed light on the shortcomings of relying on tribal knowledge, but many still struggle to alleviate their dependency. In the flow of work, few data consumers will take the time to document their knowledge. Collaboration around data takes place across disparate channels, like chat, email, meeting notes, and side conversations, making it difficult to find and share. As a result, data consumers keep asking the same questions over and over again, and work gets needlessly recreated.

Alation creates one place where everyone can collaborate and find knowledge about data. To encourage everyone to participate in sharing knowledge, Alation combines human-centered design and consumer-grade ease-of-use to make collaboration intuitive. Automated expert identification pinpoints ideal stewards and makes it easy to ask questions of the right person every time. Wiki-like articles located where people are searching for data enable knowledge to be captured from anywhere. Fully searchable, integrated conversations keep everyone connected — even across time zones — and subscription notifications alert data consumers when data sets, definitions, and articles that they care about change so that everyone is always on the same page.

As more people use the data catalog to find answers, they also commit their knowledge, making the data catalog even more valuable, further encouraging people to participate — resulting in a virtuous cycle.

Download the latest Alation State of Data Culture Report to learn more about the tribal knowledge problem and further lessons learned from the pandemic. And read about how Alation improves the day-to-day work of data consumers in this interview with Alation’s Data Science Lead, Andrea Levy.