Fiserv’s Journey to Intelligent Cloud Migration and Optimization with Alation
By Nick Jewell
Published on June 27, 2024
Key Insights
Fiserv, a leading finance technology services company, supports more than 40% of US banks and has issued 1.6 billion credit cards in the US for various companies. They stood up a modern data intelligence platform to streamline and federate data usage.
Facing a vast data landscape, the team revolutionized its data management processes with Alation, enabling secure, centralized data sharing and enhancing operational efficiency across the organization.
With Alation and Snowflake, they built 'Data Compass', to deliver metadata and business intelligence solutions (including risk profiling and customer 360) throughout the business. Data Compass provides data intelligence across 250,000+ distinct fields from Fiserv core systems to thousands of registered users. Today, they use Generative AI to convert unstructured data into enriched, documented content within the Alation platform.
Alation has been paramount in our effort at modernization in that we want all data to be described and understood by analysts, data scientists, management, and especially by end users.
How do people find data in your organization?
Data intelligence, as defined by IDC, is a ‘system to deliver trustworthy data.’ In complex data environments or larger organizations, ensuring data reliability and accessibility is crucial for making informed business decisions and driving innovation. As a result, data intelligence has become a critical component of the modern data stack in recent years.
Consider a fundamental question for data leaders: How do people find data in your organization?
This seemingly simple starting point unveils the true complexities of data management, especially when data is scattered across various departments, applications, and platforms. Finding and trusting data becomes daunting without a cohesive strategy in place, often hampering decision-making and innovation.
These questions often escalate from “Where is the data?” to more critical concerns such as “Is the data fresh?” and “How should I use it?” For analysts and governance teams, questions about data trustworthiness and regulatory compliance are not just theoretical; they directly impact strategic decisions and operational efficiency. It’s these kinds of questions that keep Chief Data Officers up at night.
Organizations need a reliable data intelligence solution to address these challenges and deliver their data modernization programs. Whether migrating to the cloud, phasing out legacy systems, or optimizing analytics, a solution must ensure data is not only accessible but also trustworthy and compliant, supporting ambitious modernization goals.
Jim Haas and Allen Goldschmidt shared the story of the Fiserv data modernization journey at Snowflake Summit 2024; what follows is a recap of that session, titled “Fiserv’s Journey to Intelligent Cloud Migration and Optimization with Alation” which explores the data management challenges faced by financial services giants and highlighting the key steps in their successful data modernization journey.
Fiserv is probably the largest company most people don't know...Some of the shocking numbers about Fiserv are that we process close to 40% of the money in the United States on a daily basis. To date, Fiserv has issued 1.6 billion credit cards in the United States for various companies.
Fiserv, a Fintech leader enabling transactions for thousands of financial institutions and millions of users, illustrates the complexity of managing vast amounts of data.
Processing transactions for banks and merchants, issuing credit cards, and running operations are just some of the backroom processes that Fiserv manages for its customers on a daily basis. This means that almost everyone in the United States interacts with a Fiserv service, whether they realize it or not.
Over 40% of US banks rely on their software, often rebranding Fiserv’s core capabilities with their own logo. Fiserv handles transactions for 3 million merchants in the US alone for retail payments, with many more globally, moving transactions and payments across markets and territories. On the consumer side, Fiserv has issued over 1.6 billion credit cards for banks, retailers, and debit providers, and over 90% of US consumers interact with Fiserv through Point of Sale (PoS) devices in stores and other retail locations.
“Fiserv is probably the largest company most people don’t know…most things are touching Fiserv,” says Jim Haas, Chief Data Architect at Fiserv and co-presenter for the session at Snowflake Summit 2024.
Beyond these core services, with over 3,000 banks’ operations handled through Fiserv, there’s a broader need for analytical oversight, which Haas’ team also handles. “Aside from transaction processing and account management, we provide analytics solutions for those banks. Our main and historical goal is to process data for money, transactions, and bank accounts… we also now help our clients to do analysis on that data,” adds Haas.
With this diverse landscape of commercial offerings, Fiserv developed a data modernization strategy to ensure data management was cost-effective for their stakeholders and governed to meet regulatory requirements, but that could remain innovative and flexible for their growing service offerings. Haas was originally hired to help Fiserv deliver this data modernization program, replacing the legacy stack of technologies that helped support these critical analytical functions, migrating to the cloud, and ensuring that the analytics computing infrastructure for Fiserv was ready and resilient for business growth.
“A good part of [data modernization] is bringing in the proper technology such as Snowflake, Alation, and other tools to make it an efficient operation,” says Haas. “We provide applications that allow [customers] to do Business Intelligence from a hosted solution. Or…they can do analytics from their location using our solutions.”
To modernize its data infrastructure, Fiserv embarked on a journey to establish a secure, unified data-sharing platform. This involved migrating clients to data and analytics services through their Snowflake accounts or hosted solutions. This transition aimed to replace fragmented data exchange methods with a centralized approach, enhancing security and efficiency.
“About three years ago, we started a modernization effort for analytics (for our customers and us),” explains Haas. “Part of that was going to the Cloud, which involves Snowflake and Alation.”
While a data modernization strategy was embraced as a strategic initiative, significant change in a regulated industry requires close attention to detail. Haas describes the cloud transformation journey as “a bit tricky because some people think it might be less safe, less secure. However, we spend a lot of time and effort to make sure it's got all the boxes checked: security, cyber, legal, contractual, and technical. We honestly know it's safer in a lot of ways.”
To streamline data exchange and leverage best-of-breed technologies like Snowflake, AWS, Streamsets, BigID, and Theom, Fiserv integrated Alation as a central data intelligence platform. Alation provided critical insights into data structure and definitions, enabling smooth data movement and communication between Fiserv and its internal and external customers.
During our cloud migration, part of this is understanding what you have, whether it’s locked up in people’s heads or in spreadsheets flying around in an email. Descriptions of data need to be coming from a central, trusted source…Alation’s central for us for that.
The data catalog emerged as a critical resource for capturing this institutional knowledge about data, built from users across the company in many different roles, including business analysts, data scientists, executives, and everyday users. “We connect Alation to our live hosted systems so that our users understand what data they’re seeing [in] their analytics solutions,” says Haas.
A quote from a Fiserv executive was inspirational throughout the journey: “Data that’s not cataloged is useless to me.” This plain-speaking message underscored efforts to ensure that every piece of data was accessible, governed, and ultimately valued through a comprehensive data catalog.
While centralizing data for analytics is often appealing to executive sponsors, its value diminishes if the organization lacks a clear understanding of what data is collected and how it can be used. Data intelligence bridges this gap, providing a broader view that makes data comprehensible and actionable.
To stakeholders at Fiserv, data intelligence is like a 50,000-foot view of what was previously hidden in an organization: it exposes what was previously hidden and helps users search and discover high-value data from across multiple source systems.
Haas’s experience with Alation helped pinpoint the key value proposition of a data intelligence solution as part of data modernization. “It's really a mashup of three things: It's a data cataloging feature where you can describe your data. It's got authoring capability, much like Wikipedia. So you can write articles and information about your data, and it's got the social elements, so you can socialize with your coworkers or others… who know about the data,” he explains.
Fiserv took a federated approach to its data catalog implementation. Subject matter experts within early-adopting business units were natural candidates for data stewards and content creators, with their deep understanding of business terminology and content. This distributed approach helped Fiserv scale its documented data catalog and governance processes across several extremely large deliverables in the early stages of the transformation program.
“We had five very large [core banking] systems that process the [transactional] banking data…those can have from 100 to 300 files [and] thousands of fields coming in from each of them, which is more than most people can keep in their heads,” Haas explains. “So, it needs to be cataloged and understood. And that's where Alation becomes very valuable.”
This need led to the creation of Fiserv’s ‘Data Compass,’ a service that delivers metadata and business intelligence solutions across various domains, including risk profiling, customer 360 views, and internal models for merchant risk and marketing. Powered by a Snowflake data warehouse, the Data Compass service provided stakeholders with comprehensive analytics to enhance decision-making and operational efficiency.
With Snowflake, Fiserv realizes the benefits of data sharing across divisions, regions, and business lines with the centralized AI Data Cloud at the core.
Behind the scenes, Alation data intelligence ensures that data movement is well understood and documented alongside the data assets' core descriptions. As Haas explains, “With Alation, we will do lineage of that data so we make sure we know what's coming from where to where. With AI governance, we need to understand all that data coming into a model and make sure everything is safe and proper.”
Data arrives in a secure landing zone in AWS before being transformed and loaded into Snowflake. Once ingested, Alation captures the critical business metadata and associated properties ready for use in BI or data sharing in client-owned Snowflake instances.
Beyond the platform itself, governance teams must oversee the data's end-to-end journey as it moves between local data centers, cloud platforms, and client-owned environments. Haas explains the value of Alation in supporting this process to help Fiserv teams understand the critical data elements necessary for compliance and control functions.
For a good model management program, we need to trust our models. We have to make sure our compliance is happy with how those are built and what data they're using. Alation provides that facility for us to make sure we know what our data is, where it comes from, and where it's going.
Fiserv’s architecture consists of encrypted source systems for core data ingestion, with Snowflake as the central repository for modeling and integrating data sources. This setup enables the creation of data products for merchant services, credit, and other business functions. An API layer then decrypts the necessary data, providing visualizations and insights that drive new revenue streams and efficiency improvements.
With over 250,000 fields in its Snowflake database, Alation’s Data Intelligence Platform enabled Fiserv teams to find, understand, and trust their data at scale. Alation’s capabilities ensured data visibility and reliability across thousands of users and millions of data points, simplifying data management and enhancing usability.
“Part of our objective is to define all the data in the whole company and every file, database, [and] table,” Haas shares. “We are on the march to do that in the different business units, and some of it is very far along…It's a multi-year effort to get that all described…In the banking area, we have petabytes of data, thousands of tables, [and] thousands of fields. It's a very large operation.”
Integrating Alation into the Agile Process
Fiserv actively involves data stewards in upcoming releases to integrate Alation with their agile software development, ensuring alignment between data management and development. Future plans include automating this integration using Alation APIs, further enhancing agility and responsiveness.
How does this curated metadata get consumed at Fiserv? Data stewards create the initial content, but new joiners to the firm are guided to explore and learn from the knowledge base immediately. Advanced users, including analysts and data scientists, are encouraged to drill deeper into critical data elements, understand compliance policies, and explore data lineage. This approach enhances their ability to derive value from data and ensures comprehensive knowledge dissemination across the organization.
“One example of how Alation helps us with search and discovery is when we bring in new people,” Haas elaborates. “I have two new people starting this week, and the first thing they'll do is go into Alation. They'll read about how [Fiserv] data works and understand the complexities of that data.”
By implementing Alation, Fiserv has minimized the risks associated with siloed institutional knowledge, which posed significant challenges when key personnel left the company. The platform’s comprehensive data catalog ensures that knowledge about data origins, transformations, and applications is preserved and accessible, which is crucial for maintaining legacy systems and mitigating personnel-related risks. Alation’s ‘popularity’ feature, derived from analyzing database query logs, has been instrumental for Fiserv. It identifies the most frequently used data elements, streamlining data movement and reducing unnecessary transformations. This focus on high-value data significantly reduces processing costs and time, leading to more efficient data management and substantial cost savings.
Fiserv adopted a federated data stewardship model to scale data governance across its vast organization. This model started with a core team of experienced stewards and content managers who developed a framework to integrate content from diverse business units into the enterprise data catalog. This strategy helped teams move beyond spreadsheets and isolated documentation, promoting consistency and accessibility in data management.
Once the federated model was established, business units contributed subject matter experts and data stewards to maintain and update their areas, aligning with Fiserv’s enterprise data vision. The successful pilot program is nearing completion, setting the stage for onboarding additional units, regions, and product lines and reinforcing a cohesive and proven scalable governance framework.
Before Alation, data was described in spreadsheets and documents. After Alation. It's searchable, it's organized, it's pointed to the source, and it's available to anyone who needs it.
Having a cloud-based data intelligence platform has also streamlined Fiserv's data modernization journey. As Haas summarizes, along with the adoption of Snowflake for data sharing with partners, working with Alation as an integrated cloud service has many additional benefits. “We moved to [the] cloud version of Alation [Alation Cloud Service], and I'm thrilled about it, actually, because it relieves us from having to manage servers, patch software, worry about operations. It's a beautiful SaaS offering”.
Alation & AI
Allen Goldschmidt, Director of Data Management at Fiserv, also took to the stage at Snowflake Summit 2024 to describe how the Fiserv data team is using Alation to explore some of the innovations around Generative AI using an “Applied AI Sandbox.”
“We can build things with [Generative AI] that can help us curate data within Alation,” Goldschmidt shared.
Fiserv is also leveraging Generative AI to transform ad-hoc content, such as spreadsheets and PDF reports, into structured, documented content within Alation. This innovation delivers seamless integration of previously untapped unstructured data sources, creating consistent and usable documentation for everyone.
Ad-hoc unstructured content is often difficult for data stewards to extract and document. Yet, it’s a perfect fit for AI's emerging strengths, such as large language models and Text Extraction. With Generative AI, innovation teams have produced a repeatable template applicable across Fiserv, standardizing the documentation process and enhancing data consistency for business sources across the enterprise.
“We have a 700-page report from a mainframe in PDF format,” Goldschmidt says. “It’s got headers and page numbers at the bottom. It’s got things that continue across pages. We’ve implemented a process to use AI to look at those pages and convert that into structured content, which we can then import into Alation. That worked out really well.”
The team can also use Generative AI to analyze Fiserv’s software code bases, generating non-technical, inherently valuable documentation for business stakeholders. This approach allows them to derive insights into data lineage from production code, providing a deeper understanding of how data flows within programming logic and enhancing cross-functional collaboration.
“We’re interested in documentation,” Goldschmidt elaborates. “We’re not the engineering team. We’re interested in getting the meaning from that code-base and applying it to [data] lineage…it can get the meaning of what you’re trying to do, where your data is going, and how it’s being transformed.”
Beyond the basics, Generative AI can interact with enterprise data using natural language queries. By integrating Alation with Retrieval Augmented Generation (RAG) processing, Fiserv aims to transform business language into precise SQL queries, unlocking valuable insights and making data exploration more intuitive and accessible.
“This is where AI and Alation shine - giving meaning to the results of that query. It’s not just tables and numbers.”
- Allen Goldschmidt, Director of Data Management, Fiserv
Key Takeaways
Data modernization is an effective digital transformation strategy that enables customers to:
Assess their legacy data landscape automatically and plan for change.
Migrate to the cloud smarter and faster, eliminating data silos and driving AI-readiness.
Optimize and scale strategic data initiatives with a lower Total Cost of Ownership.
Alation is the leading data intelligence platform for supporting each stage of this strategy, identifying, prioritizing, and migrating key data assets to the cloud. It provides data-driven insights to support continuous, incremental improvement to cloud workloads, BI rationalization, and ongoing curation, building a stronger data culture that emphasizes search and discovery, governance, data literacy, and the importance of data leadership to drive the strategy successfully.
We chose Alation because it is the industry leader…but the real reason was its user interface, [it’s] very simple to use. It speaks to non-technical users to be able to search and find meaning for the data.
Fiserv’s journey from siloed systems to AI readiness underscores the importance of incremental improvements in data modernization. Large organizations face challenges with vast data volumes and diverse environments, making comprehensive documentation impractical in a single effort. Starting with manageable projects and expanding gradually has proven highly effective in building a scalable data intelligence framework.
Haas and Goldschmidt’s experience highlights the value of starting small and curating data incrementally. Launching pilots with cooperative business units helps build early momentum, demonstrating value and securing broader support. This iterative approach facilitates stakeholder buy-in and allows ongoing refinement and scaling of the overall data intelligence strategy.
Defining your users early on is also critical. The audience for a data intelligence platform like Alation will almost certainly include a mixture of business and technical groups, each with different expectations and requirements.
“I see people in the data catalog … using it to improve their knowledge of our data, not necessarily just to perform a task, but to just educate themselves in the areas of data that we have,” Goldschmidt shares. “With Alation, we've seen a great improvement in data literacy in the company, where data that was previously locked away in silos is now available for people to learn about.”
A federated approach that embraces local subject matter experts and data stewards can really bridge the gap between expectations and reality—setting the right tone, language, and style for a data catalog glossary, capturing lineage in the right level of detail, or making sure that governance policies are explicitly defined upfront.
When it comes to building a sustainable data intelligence platform, Fiserv has embraced federation to ensure that standards are set at the enterprise level, but the individual business units have the flexibility to interpret and build out the details to meet their own specific needs, optimizing the platform for future success.
Want to learn more about Data Intelligence in Financial Services?
Discover how GXS Bank builds a strong data culture for AI-driven decision-making with Alation.
- Key Insights
- How do people find data in your organization?
- Integrating Alation into the Agile Process
- Alation & AI
- Key Takeaways
