AI-Ready Data: The Role of Data Intelligence in GenAI

The introduction of generative AI (GenAI) into the market in late 2022 resulted in a lot of hype, but behind it was also hope. Hope that science fiction-like AI technology was now in our sights, helping to significantly improve employee productivity, help with skills shortages, assist with the creation of new ideas and innovations, contribute very significant value in the economy, and drive global GDP higher.

There has been a lot of experimentation over the past year and half, with hundreds of use cases identified, but we have seen very little progress. Based on a worldwide IDC survey in April of 2024, the average number of proof of concepts executed over the past year is 37. Out of those 37, only five made it into production, with measured success rates of 68%, well below levels that IT leaders believe are required to sustain GenAI spending plans beyond 2025.¹

We are stuck in a state of AI scramble, with many concerns holding us back, including a lack of skills, cost, and most importantly, data. There is no AI without data, and control of it is the top concern of organizations that have already put GenAI applications into production. Accuracy and potential toxicity of AI-generated outcomes is the second highest concern of all organizations investing in GenAI solutions, according to IDC's survey.

AI governance begins with data governance, enabled by data intelligence. Intelligence provides the business context, relationships, quality metrics, and policies associated with security, privacy, and sovereignty classifications needed to take control of data used with AI. Data intelligence combined with model intelligence enables organizations to use data responsibly with AI. This involves understanding the provenance, quality, and classification of data, as well as the intended use and limitations of AI models. Such transparency and accountability are essential for fostering trust in AI technologies and ensuring their ethical use, meaning that the right data is being used by the right model, at the right time, and for the right reason.

Moving forward with AI

IDC is predicting that AI will have a significant impact on the global economy by the year 2030. However, organizations must be able to move past the scramble, find success in AI adoption, accelerate, and scale to the point where AI-fueled businesses are contributing to the economy. Organizations that have been successful in AI:

• Have established effective infrastructure (storage, compute) and made shared services such as security, trust, and protection available.

• Can leverage high-quality and governed data sets.

• Have strong relationships with AI software and services providers.

• Built and deployed business applications combining data and models in generating AI outcomes.

• Have people with relevant skill sets to produce and consume AI solutions and employ effective collaboration between business and IT, including legal and risk.

• Employ governance of models, data, processes, practices, and people for responsible AI within the guardrails of regulations.

• Are guided by a strategy that allows experimentation focused on production and delivery of business outcomes.

While data is only one of the dimensions that makes AI solutions successful, it is a very important one. In a worldwide IDC survey of people in or familiar with data leadership roles, namely the office of the chief data officer (CDO), 83% of respondents said that the emergence of GenAI changed their organization's data strategy, and the change is an increased focus with more than half admitting it is now the top focus of the organization.

Top strategic objectives are now to support AI initiatives, improve the quality of data and analytics products, and improve the security and privacy of data. Arguably the second two objectives are part of the first. Organizations are putting their money where their mouth is, by investing in technologies that help in making data AI-ready and looking for data intelligence solutions with AI capabilities.

This IDC survey also found strong correlations between the level of maturity in data intelligence being an enabler of AI maturity, where significantly more organizations with mature data intelligence practices also have enterprise deployments of AI. High levels of maturity in data intelligence disciplines is not only helpful in AI, but it also has positive impacts on operational and financial KPIs.

Join IDC and Alation on October 23, where we will provide more details on just what these impacts are, and how data intelligence is helping organizations get past the AI scramble and into adoption.

Citations

1. IDC, Architecting the AI-Fueled Business, 2024: Effective AI Adoption Requires a Business Operational Plan and a New Technology Operating Model, #US52576424, September 2024