Discover Accelerates Innovation with Faster Data Discovery and Active Metadata
200K
hours saved
1M
datasets cataloged
2,500+
Discover users on Alation
Challenge: Take “data-driven” to the next level
Since pioneering the first cash rewards credit card in 1986, Discover has become one of the most recognized brands in financial services. The digital banking and payment services company operates a global payment network; today, 300 million cardholders use Discover’s network at 70 million access points in over 200 countries.
“Discover is truly a data-driven company,” says Prakash Jaganathan, Senior Director, Enterprise Data Platforms. “Every decision is backed by data. Whether we offer credit to a customer, underwrite a loan, delight our customers with excellent service, or detect fraudulent transactions in seconds, our actions are supported by fresh, high-quality data.”
After more than 35 years in business, Discover has accumulated hundreds of petabytes of data spread across on-prem systems, including Snowflake and an AWS S3 data lake. As data accumulated, it became harder to strike a balance between speed to insight while preserving data integrity.
We need to deploy new financial and behavioral models faster than ever to meet customers where they’re at... But as a financial services company, we always need to balance that speed with risk mitigation.
Getting data into the hands of data scientists and analysts to create new models required a rigorous, manual ETL pipeline creation process. It took up to 30 days and required Discover’s data engineers to curate metadata, as well as to find, profile, and build quality control measures on data before they could create and run a pipeline to deliver data to internal users.
“We wanted to cut the pipeline creation timeline from 30 days to one or two days,” says Jaganathan. “The only way we could do that was to automate and power those data pipelines using active metadata.”
Objectives
Discover sought to deploy new financial and behavioral models faster to speed up time-to-market for new customer products and experiences. To accomplish this, Discover first needed to:
Leverage active metadata (to accelerate data pipeline creation)
Improve data literacy and data insights
Promote data self-service while protecting sensitive data
Gain a deeper understanding of data in Snowflake
Implementation: Enable faster data discovery and metadata-driven pipelines in the Snowflake Data Cloud
The Discover team deployed Alation, which captures and enriches metadata to enable data self-service.
“Imagine the challenge of trying to find a fit-for-purpose dataset and understand its characteristics before we implemented Alation as our enterprise data catalog,” says Jaganathan. “Alation captures active metadata, without which you cannot sift through millions of tables and find what you need.”
“Metadata enables the automation we need to build data pipelines faster and monitor the quality of our data elements,” Jaganathan continues. “Alation has become our North Star, a single pane of glass in one system for our metadata.”
Result: Faster Snowflake data insight leads to faster innovation
Today, more than 2,500 users across the global workforce employ Alation to find, use, and enrich the metadata in a million-plus cataloged tables. They can find dataset information, including those in Snowflake, in as little as 15 minutes and trust it. The productivity gains and faster time to insights lead to more time for innovation – and in Discover’s case, that equates to 200,000 hours saved.
By improving the speed at which we are able to acquire and use data, we can turn that saved time into product innovations that help us remain No. 1 in customer satisfaction and delivering personalized customer experiences.
Actionable metadata leads to faster pipeline creation
Prior to Alation, data engineers at Discover spent a good amount of time on data discovery and metadata curation. Today, technical engineers have the metadata they need to generate pipelines and stay apprised of metadata changes.
“We have improved our data engineers' developer experience,” says Jaganathan. “They have more time to understand the data that they are provisioning instead of just being the plumbers who are moving data, as in the past.” This has empowered engineers to build domain expertise, so they can more quickly spot and solve data quality issues.
Self-service boosts insights and organizational data literacy
“Data documentation used to be a laborious process,” says Jaganathan. “There was no excitement in documenting data, and we were not able to collect consistent quality metadata before Alation. Today, the data discovery process takes as little as 15 minutes instead of up to 2 days.” Alation also helps boost data literacy across the organization as users can easily document datasets and contribute quality metadata through articles and conversations. The result? Discover is no longer reliant on tribal knowledge. "People are excited to use Alation’s friendly user interface.”
AI and Alation: The future of data at Discover
Discover intends to increase its use of AI and ML to support strategic decision-making. “The key to using AI and ML to make decisions is having context for the data,” states Jaganathan. “Alation can ensure that the datasets are labeled with accurate metadata, which tells the machine how to understand the context around the data it’s looking at. Only then can it answer our questions in a way that’s meaningful.”
