Driving Data Catalog Adoption

In a recent blog, titled Collaboration and Crowdsourcing with Data Cataloging, I discussed the importance of participation by all data stakeholders as a key to getting maximum value from your data catalog. Many organizations, however, find data catalog adoption—getting people to participate—to be among the biggest challenges to data catalog success. Adoption is challenging, but understanding the causes of resistance and developing an adoption plan help to overcome those challenges.

Understanding the Adoption Challenges

When planning for data catalog implementation, the human and cultural dimensions of data cataloging are often overlooked or subordinated to the process and technology dimensions. A typical data catalog implementation process begins by defining the business and technical case, proceeds through technology selection and installation, then moves on to data discovery and populating the metadata catalog. (See figure 1.) This build-it-and-they-will-come approach fails to engage people to actively use the catalog.

The final step—use the catalog—often doesn’t happen at the level expected for a variety of reasons. (See figure 2.) Predominant among those reasons:

• New Methods as a Barrier—It is human nature to be anchored by “the way that we’ve always done it.” The shift to new ways of doing things pushes people away from the familiar and comfortable. Self-service data consumers may resist the data catalog and continue to rely on personal networks and tribal knowledge because it is what they know how to do. Using the data catalog requires them to learn new things, which can seem time-consuming and disruptive for busy people.

• Culture Shift—Data cataloging is most successful in a culture of data sharing, knowledge sharing, and collaboration. Behaviors such as the “my data” mentality, territorialism, and knowledge hoarding are signs of an unhealthy culture that is a barrier to becoming a data-driven organization. A healthy data culture encourages collaboration and sharing, and discourages the unhealthy behaviors. Participation is a key element of data culture—participation at all levels. Leadership visibly invests in data management and in growing data literacy throughout the organization. Staff are encouraged and incentivized to access and analyze data and to share their knowledge about working with data and share the insights that they derive from data.

• Data Literacy—Many line-of-business people have responsibilities that depend on data analysis but have not been trained to work with data. The skills to get from data to useful information—data selection, data understanding, data preparation, data analysis, data visualization, and data storytelling—are not native and natural for them. Their tendency is to do just enough data work to get by, and to do that work primarily in Excel spreadsheets. The data catalog and access to abundant data often feels more like hazard than opportunity to these people.

• Motivation—Changing how you work and learning to use the data catalog can seem intimidating, time-consuming, or simply put you out of your comfort zone. Most people will resist change until they see how it benefits them personally. What’s-in-it-for-me (WIFM) is a typical response, especially when asked to do new things such as participate in metadata crowdsourcing and post ratings and reviews of datasets. WIFM is a major influence in resistance to data sharing, resistance to knowledge sharing, reluctance to participate in collaborative curation, and reluctance to post ratings and reviews.

Figure 2 – Barriers to Data Catalog Adoption

Developing a Data Catalog Adoption Plan

Overcoming the challenges of data catalog adoption takes more than patience and persistence. You’ll need a plan that recognizes adoption as a process that unfolds over time—not an event that happens at a point in time. The plan must address people, organization, culture, and training as data cataloging factors to be synchronized along a time line.

• People – Which data stakeholders will you bring on board at what times?

  • Start with power users and high-literacy data consumers. They’re likely to be the most enthusiastic, and they’ll give you good feedback and an opportunity to fine tune before bringing others on board.

  • Look for opportunities to create value and demonstrate success. When you can showcase successes in time savings, cost savings, and analysis quality that lead to real business impact don’t hesitate to blow your own horn. Visible successes will have others wanting to get on board.

  • Develop advocates and business champions. When data catalog advocacy comes from an IT department it is likely to be viewed with some skepticism. Advocacy comes from line-of-business data consumers is certain to have greater impact.

• Organization – What organizational changes should be made at what times?

  • When will you need to formalize data curation roles and designate domain and lead data curators? Will curation be a full-time role for anyone? Will data stewards assume curation responsibilities? How will data curation be funded?

  • When and how should data governance evolve and change? What changes are needed to be compatible with autonomy and agility interests of self-service data consumers? How does the data catalog support data governance? What new governance activities might data cataloging require?

  • When should you consider data coach as an important role? Data coaches are data fluent line-of-business leaders in the use of data who share their knowledge, skills, and best practices with others. Will you increase data literacy by designating skilled data consumers as coaches to help less skilled people improve their abilities for working with data? How will data coaches fit into the organization? How should they be recognized or incentivized?

• Culture – What culture shift catalysts will you introduce at what times?

  • What kinds of collaboration and crowdsourcing incentives will you use? Is public recognition of frequent contributors enough? Will small awards such as preferred parking spaces or Starbucks cards be enough? Can you create competition to contribute through gamification?

  • What kinds of self-service analysis and reporting incentives will you use? How will you identify those who would benefit from self-service but aren’t participating? What will it take to get them on board?

  • What kinds of data sharing and knowledge sharing incentives will you use? How can you reward sharing? How can you discourage territorialism and hoarding? How can executives and corporate leadership communicate the vision and expectation for a culture of sharing?

• Training – What training is needed by whom and at what times?

  • Who needs data literacy training? In what specific areas– data selection, data understanding, data preparation, data analysis, data visualization, and data storytelling—is training needed? Who can provide the training? When and how frequently should it be offered? For what groups will scheduled classroom training be effective? For whom will individual self-paced training work well? Where can you get advantage from on-the-job learning with data coaching?

  • How will your primary data curators—lead and domain curators—learn the skills of data curation? Who can teach them and how will training be delivered?

  • Who will need data catalog tool training? What tool functions and features do they need to learn? Who will deliver the training? When and how frequently should training be offered? How will you address ongoing training needs to refresh and for new employees?

These are the key questions that are central to a data catalog adoption plan. Use them as a starting place and adapt to the unique circumstances and challenges of your organization. Planning for adoption is every bit as important as planning for technology implementation and metadata discovery. To truly succeed with data cataloging you must address the human and cultural dimensions too.