Reshaping Business Processes with AI: Insights from Raza Habib, CEO of Humanloop

Large Language Models (LLMs) are revolutionizing business operations, transforming how organizations approach automation, decision-making, and efficiency. In a conversation on Data Radicals, Raza Habib, co-founder and CEO of Humanloop, shared his insights on the evolving landscape of AI, including prompt engineering, fine-tuning, AI agents, and the role of data product managers.

Why AI demands a redesign of business processes

Humanloop is an LLM evaluation platform that enables businesses to iterate quickly on customized AI applications through prompt engineering and fine-tuning. What do the organizations that use this technology successfully have in common? They build interdisciplinary teams of experts, including nontechnical professionals. This marks a shift in how software is designed. Indeed, as LLMs grow more powerful, organizations must rethink traditional software development and business workflows.

Prompt engineering—crafting and refining natural language instructions for AI models—has become a core aspect of AI development. Habib explains:

"People used to customize these models by labeling annotated data, and now they write prompts. Prompt engineering has become a big part of AI development, and prompts blur the line between data and code."

Companies like Gusto, a payroll and HR platform, employ Humanloop to improve its internal customer support AI and report-building capabilities. They initially allowed engineers to experiment freely with AI features. However, as AI became more integral, they adopted a centralized strategy to ensure effective AI deployment. By implementing evaluator scores and feedback mechanisms, today Gusto refines AI-driven solutions for efficiency and accuracy.

The data product manager’s evolving role in AI development

The role of data product managers (DPMs) is undergoing a significant transformation in the era of AI, particularly with the advent of LLMs. Traditionally, DPMs focused on managing reusable, governed data assets to drive business value. However, as AI capabilities expand, their responsibilities now encompass integrating analytics and AI into data products, ensuring these technologies align with business objectives and user needs.

Organizations differ in their definitions of data products. According to a survey highlighted by MIT Sloan Management Review, 48% of respondents include analytics and AI capabilities within the concept of data products, while 30% view them separately, reserving the term 'data products' for reusable data assets alone. Only 16% don't consider analytics and AI in a product context at all. The key takeaway is the necessity for organizations to maintain internal consistency in how they define data products to avoid confusion.

Habib emphasizes the importance of involving domain experts in AI development. He cites the example of Filevine, a contract lifecycle management company, where actual lawyers participate in writing prompts, reviewing outputs, and investigating AI performance. This collaboration ensures that AI models are not only technically sound but also contextually relevant and effective.

In essence, the evolving role of data product managers in AI development requires a multidisciplinary approach, clear organizational definitions, and active collaboration with domain experts to create AI-driven data products that deliver tangible business value.

Measuring the economic value of generative AI

Quantifying the ROI of generative AI requires robust evaluation frameworks and performance metrics. Humanloop’s observability tools help trace data through AI workflows, assessing impact through systematic iteration. Gusto has used Humanloop to deliver an internal product that delivers measurable ROI: a report builder. Habib elaborates:

“Gusto is a payroll company; accountants will go into their software and want to be able to build reports. It used to be something that took someone maybe an hour or two to build a complicated report based on the data they have. And they've built an AI copilot that can guide people through that process. And in building those things, they had to do a lot of prompt engineering. They had to get feedback and review from the customer support people to understand how well things are working."

By creating evaluator scores and leveraging both human and code-based feedback, Gusto has been able to systematically improve its systems over time. This rigorous, metric-driven approach not only optimizes operational efficiency but also provides a clear ROI for generative AI investments.

Customizing LLMs: Prompt engineering vs. fine-tuning

Organizations looking to tailor LLMs to their needs have two primary approaches: prompt engineering and fine-tuning. Prompt engineering enables fast iteration without requiring extensive retraining. As Habib notes:

"It's much faster to change a prompt and see the impact than to retrain a model. It’s often sufficient for customization."

Fine-tuning, however, is beneficial in specific scenarios:

• When tasks require outputs that are difficult to describe explicitly (e.g., maintaining a particular tone of voice).

• When optimizing large models for lower cost and latency by training smaller, more efficient versions.

Both prompt engineering and fine-tuning offer valuable pathways for customizing LLMs, each suited to different use cases. While prompt engineering provides a quick and flexible way to adapt models with minimal overhead, fine-tuning is essential for tasks requiring nuanced output control or efficiency optimizations. Organizations should evaluate their specific needs to determine the right balance between these approaches, ensuring they maximize AI performance while maintaining cost-effectiveness.

Real-world applications of LLMs

How are organizations leveraging tools like Humanloop? Today, many businesses are leveraging LLMs to enhance operations. Here’s a breakdown by industry:

• Customer Support: Moveworks automates IT support through conversational AI.

• Finance: The Royal Bank of Canada uses AI for secure, productivity-enhancing applications.

• Healthcare: AI aids in diagnostics and personalized treatment planning.

• Legal Industry: Law firms automate contract reviews and legal research.

• Education: Platforms like Khan Academy integrate AI-driven personalized tutoring.

AI agents: Orchestrating inter-organizational ecosystems

By 2026, over 70% of enterprises are expected to allocate a portion of their AI budgets to AI agent development. AI agents have the potential to automate workflows such as pricing optimization, contract negotiation, and compliance monitoring, streamlining operations while overcoming organizational silos.

Unlike standalone LLMs, AI agents integrate decision-making frameworks with language generation to manage complex, multi-party processes. These autonomous software entities are poised to drive efficiency across industries, facilitating collaboration between diverse stakeholders.

Conclusion

Raza Habib’s insights highlight AI’s role in reshaping industries. Whether through optimizing workflows with prompt engineering, orchestrating ecosystems with AI agents, or evolving data product management, the future of AI is as much about collaboration and processes as it is about technology. Organizations that embrace structured evaluation frameworks and AI-driven transformation will be best positioned to succeed in the evolving landscape of generative AI.

Curious to learn how a data catalog can help you leverage the power of AI? Book a demo with us to learn more.