Article | RSM Labs

Stop prompting. Start architecting.

AI is breaking the enterprise operating model

March 11, 2026
#
Artificial intelligence RSM Labs

Artificial intelligence isn’t revolutionizing how businesses operate—it’s exposing their deepest cracks.

Everyone is chasing prompts and churning out more summaries, analyses and presentations than ever, but real progress remains elusive. Beneath the excitement, the output surge is a mirage, masking fundamental organizational inertia that dashboards can’t reveal.

The operating model underneath all that activity hasn't changed at all.

AI isn’t failing because the models are weak. It’s failing because the enterprise is trying to bolt a probabilistic engine onto a deterministic machine. The result is a system that looks faster on the surface but strains under the weight of its own acceleration.

A closer look

The RSM Middle Market AI Survey 2025: U.S. and Canada found that generative AI penetration had climbed to 91% among respondents, but just 25% of adopters had fully integrated generative AI across their operations (another 43% had integrated it into some processes).

This is why so many leaders feel discomfort but can’t pinpoint why. AI speeds up work in isolated pockets, but the enterprise absorbs that speed as friction, not progress.

It’s the leadership equivalent of pressing the gas while the parking brake is still on.

AI isn’t failing because the models are weak. It’s failing because the enterprise is trying to bolt a probabilistic engine onto a deterministic machine.

The illusion of progress

Generative AI creates content at a pace no human system was designed to handle. It drafts content in seconds, yet the review cycle still takes days. It produces a forecast instantly, but now the audit, validation and sign-off require an enhanced, multistep human-in-the-loop sequence.

This is the illusion of progress. The output accelerates, but the organization does not.

Under the surface, something else happens. When machines take over first-draft creation, humans don’t disappear; they migrate into their new roles of continuous reviewers, validators, editors and risk absorbers. Every AI-generated asset triggers a new chain of oversight.

Simply put, prompting amplifies activity, but architecture determines capability.

Instead of trying to fit AI into the current architecture of how humans work, leaders need to re-architect the process to be AI-driven, with employees thinking at a systems level—higher than their hands-on work. Many companies are not currently equipped for this, which leads to mixed or poor results.

This is the part most organizations underestimate. Leaders assume the bottleneck is when generating the work. It’s not. The bottleneck is in decision making, validation, compliance, risk management, accountability and data trust. And none of those accelerate automatically when you introduce a generative engine.

Simply put, prompting amplifies activity, but architecture determines capability.

Where the operating model cracks

Cracks in the operating model tend to show up in the same places, regardless of industry.

1. The rework burden

AI multiplies the volume of drafts, options, versions and alternatives. Each requires validation. The organization becomes a factory of checks. This obstacle is a “slowness tax” on the organization because the validation and second-guessing can slow the decision-making process. 

2. Human-in-the-loop overload

The idea of humans supervising AI sounds good on paper. In reality, it often funnels more decisions to already overloaded leaders, turning them into the last mile for every workflow. 

3. Deterministic controls meeting nondeterministic systems

Finance, risk and compliance teams expect repeatability. However, AI does not always produce the same answer twice. That variance forces organizations to create new layers of oversight that slow everything down.

4. Governance that moves too slowly

Pilots succeed because they’re insulated with clean data. Production fails because real data is messy, siloed, duplicated, stale or contradictory. AI magnifies those weaknesses.

5. Fragmented adoption

Without architectural guardrails, every employee develops their own AI stack: their preferred models, prompting styles, tools and workflows. In this shadow AI environment, the enterprise becomes a patchwork of personal productivity hacks rather than a cohesive system.

These aren’t technology problems. Instead, they’re operating model problems.

A closer look

Data quality issues emerged as a key concern in RSM’s 2025 AI survey, as 41% of respondents who experienced AI implementation issues said poor data quality was a key barrier to successful deployment.

Why prompting is a false positive

Prompting is seductive because it’s easy. It gives leaders a rush of early wins with faster deliverables, lighter lifts and visible productivity. But prompting alone doesn’t create institutional capability. It creates localized efficiency in a globally inefficient system.

The deeper issue is that prompting has little to no switching cost.

Employees can hop between tools, models and patterns with minimal structural commitment. In this scenario, experimentation skyrockets and enterprise cohesion evaporates.

Leaders often mistake activity for integration. They see increased activity on dashboards and assume progress, while nothing fundamental has changed about how the organization makes decisions, routes work, manages risk or creates value.

The result is an enterprise with dozens of micro-optimizations and no macro-transformation.

Leaders often mistake activity for integration.

Architecting: The shift from output to operating system

Architecting is not a technical exercise. It’s an organizational one.

It requires rebuilding how decisions are made, how work flows, how data moves and how the enterprise defines accountability when machine actors enter the system. It is the difference between AI-assisted tasks and AI-powered enterprise capability.

When organizations architect instead of prompt, AI stops being just a productivity feature and becomes an infrastructure operating system.

Architecting unlocks five structural advantages:

1. AI-native workflows

Instead of AI-injected versions of legacy processes, workflows are restructured so machine judgment does the heavy lifting and human judgment is reserved for inflection points.

2. Clarity on decision rights

What decisions can AI make without human review? Which require supervision? Which must remain fully human? Without clarity, organizations default to universal oversight, which quietly negates any promised speed.

3. Embedded governance

Governance cannot be episodic. It must be continuous, observable and automated. If humans are still the primary governors of AI output, scale collapses under their review cycles. 

4. Economic discipline

Token usage, data access and compute consumption all come with variable cost curves. Architecting brings these costs under control before adoption balloons into a FinOps surprise.

5. A unified platform instead of tool sprawl

Architecture replaces the chaos of shadow AI with a consistent way of integrating models, securing data, orchestrating workflows and scaling capability.

When organizations architect instead of prompt, AI stops being just a productivity feature and becomes an infrastructure operating system. 

A closer look

In the RSM US Middle Market Business Index Special Report: Cybersecurity 2025, 34% of smaller middle market firms surveyed indicated that AI governance steps are not yet in place.

What this forces chief financial officers to confront

AI introduces a new kind of budget challenge. It doesn’t behave like software with fixed licensing. It behaves like cloud computing … elastic, usage-based and hungry.

Without architecture, CFOs face:

  • Expanding compute costs with no visibility
  • Shadow AI spending hidden in corporate credit cards 
  • Vendor lock-in risks as workflows embed specific models
  • Difficulty proving return on investment beyond anecdotal productivity gains
  • Rising regulatory exposure if data governance isn’t designed into the system

Instead of asking “How much are we spending?” CFOs need to ask, “Where are we embedding structural costs?

The only way to answer that is through architecture. Not prompts.

What this forces chief information officers to confront

CIOs sit at the pressure point between innovation and risk. AI turns that tension into a daily operational reality.

Without architectural discipline, CIOs face:

  • Proliferating shadow tools
  • Permanent pilot paralysis 
  • Weak data boundaries 
  • Model integrations that multiply risk exposure
  • Irreversible vendor dependence
  • Governance models that can’t keep up with machine-speed decisions

The best CIOs are encouraging AI adoption as much as they are fighting AI disarray.

A closer look

In RSM’s 2025 cybersecurity special report, 20% of U.S. respondents said their firms lack AI governance, meaning CIOs in those organizations have limited oversight on what data is going into which AI tools. This raises data leakage and cybersecurity risks.

Architecture gives them what they need. It provides a system for scale, not a patchwork of experiments. It establishes patterns for integration, observability, security and governance that prevent the enterprise from fracturing into incompatible AI pockets.

This is the shift from controlling AI usage to designing the environment in which it operates.

What architecting actually requires

Architecting is not about better prompts, bigger models or “more AI.” It is about committing to enterprise‑level design decisions.

It requires:

  • Redesigning workflows around machine participation
  • Building a clean core
  • Defining accountability before, not after, automation
  • Moving to contextual containers and skills instead of relying on “one-shot prompts”
  • Treating data governance as infrastructure, not policy
  • Embedding secure-by-design guardrails
  • Designing for nondeterminism, variance, drift and decay
  • Building transparency into every layer of the AI lifecycle
  • Training the organization to operate at the speed of machine output vs. human habit
  • Architecting human-in-the-loop escalation protocols and hyperscaler neutrality

Executive action items

1. Stop counting prompts. Start counting workflows.
If AI doesn’t shorten end‑to‑end decision cycles, it’s not driving transformation.

2. Redesign the work before you automate it.
Legacy workflows wrapped in AI still behave like legacy workflows, but with different errors and necessary human interaction.

3. Establish governance that is architectural, not procedural.
Manual review will never catch up with machine output.

4. Expect elastic cost curves. Manage them early.
The FinOps bill always arrives. The only questions are when and how prepared you are.

5. Align CFO and CIO incentives before you deploy anything.
AI breaks silos by its nature. Your operating model has to break with it.

The takeaway: Commitment is key

Prompting is how you experiment, but architecting is how you scale. 

Organizations that stay in prompting mode will drown in rework, cost and complexity. Conversely, the ones that commit to architecting will transform AI from a novelty into an operating advantage.

AI is breaking the operating model—not through a single dramatic failure, but through a steady buildup of strain the current system was never designed to carry.

Prompting is how you experiment, but architecting is how you scale.

The sooner leaders focus on architecting, the faster AI becomes what everyone hoped it would be: an accelerant, not an anchor.

Ready to get started? The time is now to build an AI strategy that drives real results. Download RSM’s middle market AI playbook for details on successfully increasing efficiency, insight and productivity.

Deeper insights for curious leaders

Contact our business strategy and operations professionals

Complete this form and an RSM representative will be in touch shortly.