RSM fireside chat: Middle market AI trends and challenges

The emphasis on successful artificial intelligence adoption in businesses is intensifying. Organizations seek to leverage the technology’s potential to increase efficiency, gain insight and improve the user experience across a range of functions. In fact, the majority of middle market businesses report using AI to varying degrees, according to new RSM research. The RSM Middle Market AI Survey 2024: U.S. and Canada found that 78% of executives at midsize companies use AI formally or informally, while 77% have adopted generative AI solutions. Use cases for generative AI are broad and led by quality control and customer service.

In addition, the RSM survey found:

• Only one-fifth of executives feel their firms have achieved complete AI integration.
• Eighty-five percent agree that generative AI exceeds their expectations.
• Nine in 10 want to increase their AI budgets.

In response to this survey data, RSM hosted a panel of chief data officers to share findings and understand real-world challenges from data executives and solutions playing out in the middle market in real time.

Fifty-four percent of RSM survey respondents found implementing generative AI more difficult than expected. Panelists highlight challenges in relation to generative AI adoption, including:

• Proof of concept projects not reaching production at scale
• Keeping up with an ever-evolving spectrum of AI tools
• Lack of high-quality data and robust data infrastructure

Panelists stressed that AI use would only be productive or provide accurate results with good data. “There are a lot of challenges around data quality and data integration,” said Tarun Sood, chief data officer for American Century Investments. “If your data is bad, AI is just going to magnify it and show how bad.”  

Organizations often lack the internal skills necessary for successful AI deployment and frequently require external assistance. Furthermore, with the complexity of AI models, organizations struggle to comprehend insights and analytics fully and adapt the most effective solution(s). The need for continuous learning and AI integration will be important ongoing objectives for firms trying to stay current.

While new AI tools such as AutoML and GitHub Copilot allow companies to build near-effortless prototypes to improve processes, companies face significant challenges in moving prototypes to full implementation due to a lack of planning and resources. To avoid logjams, businesses should quickly assess data viability before committing to full-scale projects and create an efficient strategy to support the AI lifecycle.

“There is this expectation of a silver bullet with immediate ROI,” said RSM Principal George Casey, noting that upskilling around AI use requires careful planning within an organization.

Panelists strongly emphasize that organizations should develop processes to optimize the transition from AI ideation/experimentation to full-scale production. Additionally, focusing on fundamentals, such as where there is a good problem to solve or how we can provide better customer service that entails a strategic approach, will be pivotal in helping organizations harness the potential of AI technologies.

Casey reminds organizations implementing AI “doesn’t have to be a home run. Find out where there are good singles and doubles to get your organization upskilled, shifting to data-driven and AI first."

The key to successful AI integration is good leadership. The evolving role of a modern CDO requires the alignment of AI data analysis to an organization’s broader vision, mission and business goals, requiring multifaceted strategic thinkers who also possess technical proficiency.

George Casey emphasized the importance of the CDO’s understanding of the dynamics within an organization and ensuring that key stakeholders recognize and support AI’s role in accelerating growth. “It’s all about curating an environment and understanding data as a product,” he said, noting that AI will be used to serve that broad constituency, which can include employees, suppliers and customers.

Cross-functional teaming is necessary to get the most out of AI technology, said Sood, underscoring the need for a CDO to have good soft skills that can coalesce disparate groups. “Teams across functions and lines of business are working together and collaborating more closely to build solutions, indicating a significant shift within the dynamics of firms,” he noted. “There is definitely a more unified approach between the businesses.”

Being able to take an enterprise view of AI’s deployment is essential. “The use of AI should align with a firm’s strategic priorities, and any investment in AI technology should directly contribute to the business’s goals and return on investment, especially during budget cycles,” said AltaMed Chief Data and Analytics Officer Greg Townsend.

Securing board buy-in for data governance and strategy can be challenging, especially when topics like data quality don’t seem as immediately captivating as AI. By aligning corporate priorities with AI strategies, however, CDOs can demonstrate how a solid data infrastructure is essential for the long-term success of AI initiatives.

AI use cases for immediate benefit

While AI technology has the potential to tackle sophisticated business processes ranging from predictive analytics to financial modeling and market trend analysis, companies can see an immediate benefit from the technology by applying it to preexisting processes, including:

• Automating routine back-office functions such as payroll and accounts payable
• Assisting with documentation review
• Deploying AI-enabled chatbots and virtual agents to enhance customer service
• Ideating and developing content
• Assisting with coding (e.g., GitHub Copilot)

These are just a sampling of the immediate value that AI solutions can provide. As the technology matures, new use cases will emerge that create opportunities to enhance the user and customer experience and strengthen efficiency, creativity and insight.

AI governance and risk management

Establishing careful AI regulation within an organization is crucial for risk mitigation. It includes the following considerations:

• Governance
• Legal parameters
• Bias and fairness in AI models
• Cross-functional alignment
• Regulatory compliance

"Transparency is essential in AI,” Casey noted. “No model is entirely accurate; each operates with a certain level of probability."

Without an effective AI governance approach, an AI strategy may not deliver on its expected value, with a higher probability of inaccurate outputs and cyber and risk vulnerabilities. While risk management and governance are not often the first things that come to mind when considering an AI strategy, they are essential elements to success. 

AI has rapidly emerged as an integral facet of a comprehensive middle market technology strategy. However, developing and implementing a successful AI approach can be complex, with many elements to consider, as well as evolving risks and challenges. Ultimately, AI can significantly enhance an organization’s efficiency, decision making and overall performance, but effective leadership along with proper execution through a clear framework backed by good data are important for programs to reach their full potential.

View our AI fireside chat webcast on demand for more information on how to build a successful AI strategy while addressing potential obstacles and risks that could impede success.