Author: Bob Walder

Why Enterprises Need a Better Way to Evaluate AI Security

Posted on by Bob Walder

Enterprise adoption of AI has moved well beyond experimentation. Large language models, retrieval-enabled assistants, and increasingly autonomous agents are being connected to internal data, operational workflows, and decision-making processes across the business. That shift creates opportunity, but it also creates a problem: many organizations are deploying AI faster than they are developing the security discipline required to control it.

For security leaders, the challenge is no longer theoretical. It is practical and immediate. Questions about data exposure, prompt manipulation, delegated authority, policy enforcement, resilience under stress, and auditability are now showing up in real buying decisions. Yet the market for AI security products is still immature. Terminology is inconsistent. Product categories blur together. Demonstrations often show only narrow success cases. What buyers need is a more defensible way to decide which controls are meaningful, which are incomplete, and which claims can actually withstand scrutiny.

In March NSS Labs published two-part white paper series on enterprise AI security.

The first paper, AI Security Beyond the Model: What Enterprises Need to Care About — and Why, establishes the core argument that securing the model alone is not enough. In real deployments, the primary risk does not sit neatly inside the model. It emerges in the systems surrounding it: the data sources it can access, the tools it can invoke, the permissions it inherits, the policies that govern it, and the visibility the organization has into how it behaves. The paper examines these issues through the lenses of input integrity, output risk, resilience, policy governance, agentic behavior, observability, and GRC.

The second paper, Evaluating Enterprise AI Security: Questions Every Buyer Should Be Able to Answer, takes that architectural and governance framing and turns it into an evaluation framework. It is aimed squarely at CISOs, enterprise buyers, GRC leaders, and security architects who need to assess vendors under real-world conditions. The paper lays out the questions buyers should ask, the warning signs they should watch for, and the criteria they should use when comparing AI security controls for production environments.

A central theme across both papers is the role of runtime guardrails. These are the controls that operate outside the model, where enterprise risk actually materializes. They are responsible for enforcing policy, constraining access, mediating tool use, reducing data leakage, and generating the evidence that security and governance teams will need when something goes wrong. However sophisticated a model may be, those external controls are what determine whether AI can be operated safely at enterprise scale.

Another important point in the series is that AI security cannot be separated from governance. Enterprises do not just need controls that block obvious abuse. They need controls that can be explained, tested, monitored, tuned, and audited. They need to know what decisions were made, which policy triggered them, what data was accessed, which tools were invoked, and whether authority was properly constrained at every step. In short, they need an approach to AI security that is operationally credible, not merely technically impressive.

This work also reflects a broader need in the market. AI security is moving quickly, but rigorous evaluation standards are not keeping pace. That gap makes it harder for buyers to distinguish between sound engineering and attractive narratives. It also makes it harder for vendors with genuinely strong controls to prove their value in a disciplined way. Establishing clearer expectations for testing, transparency, and validation benefits both sides of the market.

NSS Labs believes that enterprise AI security needs the same kind of disciplined, evidence-driven thinking that has long been applied to other areas of cybersecurity. That means moving beyond architecture diagrams and product claims toward repeatable evaluation, realistic testing, and accountable governance. These two papers are intended to help establish that foundation.

This series is not the end of the work. It is the start of a larger effort to define how AI security controls should be evaluated and, ultimately, how they should be tested under independent conditions. As AI systems become more embedded in enterprise operations, the organizations that succeed will not be the ones that move fastest without discipline. They will be the ones that pair innovation with control, and ambition with evidence.

NSS Labs has already begun testing AI Protection Systems (AIPS). If your organization is evaluating AI security products, these papers are intended to provide a clearer starting point: what matters, what to ask, and what good should look like.

If you would like to see an AIPS product tested, now is the time to reach out to us.

Read the white papers:

AI Security Is Moving Fast. Evaluation Isn’t. That’s a Problem.

Posted on by Bob Walder

AI adoption in the enterprise is not creeping forward. It’s sprinting.

In many organizations, it’s closer to “build first, figure out the risk later” than many would care to admit. New copilots, internal assistants, and increasingly autonomous agents are being wired directly into data, workflows, and decision-making processes. The business sees speed and advantage. Security sees… well, a growing list of unanswered questions.

Here’s the uncomfortable truth: most enterprises are making consequential AI security decisions without a reliable way to evaluate whether the controls they’re buying actually work.

That’s not a knock on buyers. It’s a gap in the market.

Right now, AI security is full of confident claims, polished demos, and tidy architecture diagrams. But those don’t tell you how a system behaves under pressure, how it fails, or whether the controls hold up when someone actively tries to break them. And if I have learned anything in my decades at NSS Labs testing this stuff, it is that if there is a vulnerability, it will be exploited.

So we decided to take a step back and ask a very simple question: what does “good” actually look like?

That question led to a two-part research series from NSS Labs focused on how enterprises should think about—and evaluate—AI security.

The first paper, “AI Security Beyond the Model,” makes a point that sounds obvious once you say it out loud: the model is only part of the problem. The real risk lives in everything around it—the data it can touch, the instructions it can be manipulated with, the tools it can call, and the permissions it inherits. If those aren’t controlled properly, even a well-aligned model can do the wrong thing, quickly and at scale.

The second paper, “Evaluating Enterprise AI Security,” takes that idea and turns it into something buyers can actually use. It lays out the questions that should be asked in every evaluation, the red flags that should raise eyebrows, and the criteria that help separate meaningful controls from wishful thinking.

A big part of that conversation centers on runtime guardrails.

Not the model. Not the training process. The controls that sit around the model and determine what actually happens in production.

These are the mechanisms that enforce policy, limit access, constrain agent behavior, and—crucially—provide us with a solid trail of evidence. Because sooner or later, something will go wrong. When it does, “the model decided to do it” is not going to satisfy anyone in legal, compliance, or the boardroom.

If that sounds a bit like traditional security thinking, that’s because it is. We’re just applying it to a new class of systems that behave in less predictable ways.

There’s also a broader point here. AI security is evolving quickly, but the way we evaluate it hasn’t caught up. Without clearer expectations, buyers are left comparing apples to… well, marketing slides. And vendors with genuinely strong capabilities don’t have a consistent way to prove it.

That’s not a healthy place for the industry to be. So to address the problem, NSS Labs has been working hard behind the scenes with the major players in the AI Protections System (AIPS) space to define a new and comprehensive test methodology that will allow us to apply the NSS Labs usual stringent testing and evaluation approach to this emerging market.

Our goal with this work isn’t to declare winners or define a single “right” approach. It’s to raise the bar on how these systems are assessed. To move the conversation from “this looks good in a demo” to “this holds up under scrutiny.”

Because AI isn’t slowing down. If anything, it’s accelerating. And the gap between deployment and accountability isn’t going to close on its own.

If we want AI to be trusted at enterprise scale, we need to get serious about how we evaluate the controls that make it safe to use.

That starts with asking better questions—and expecting better answers. And the way to do that is through truly independent third-party testing. A brand-new comprehensive test and validation methodology, published today, evaluates AIPS products across the core areas that matter most in real enterprise deployments, including protection against prompt injection, prevention of harmful or unauthorized output, evasion techniques, resilience under stress and adverse conditions, policy and filter efficacy, security of agentic behavior and tool invocation, observability and auditability, and performance impact.

Each test dimension is designed to represent realistic risks that enterprise customers may encounter when deploying AI systems connected to users, enterprise data, tools, APIs, and business processes. The goal is to provide enterprise buyers, security leaders, and product vendors with a clear, repeatable, and technically rigorous basis for measuring how effectively an AIPS performs under conditions that reflect real-world use and abuse scenarios

If you’re working through these challenges now, the two white papers are designed to give you a practical starting point: what matters, what to test, and what good should look like when you find it. The goal for the AIPS test methodology is to provide enterprise buyers, security leaders, and product vendors with a with a clear, repeatable, and technically rigorous basis for measuring how effectively an AIPS performs under conditions that reflect real-world use and abuse scenarios.

Once testing is completed later this year, the final reports will provide incredible insight into how security vendors are addressing these problems.

The AI Automation Arms Race: Why Defense Is Not Symmetrical

Posted on by Bob Walder

The security industry likes to tell itself a comforting story: as attackers adopt artificial intelligence, defenders will respond in kind, and the balance of power will remain roughly equal. AI on both sides, the thinking goes, should cancel out.

This assumption is wrong — and potentially dangerous.

As with most other areas of security, the use of AI in practice is deeply asymmetrical. Attackers benefit disproportionately from automation, while defenders struggle to translate AI adoption into meaningful risk reduction. The result is not an arms race between equals, but a widening gap between the speed at which attacks evolve and the pace at which enterprises can govern, understand, and respond to them.

Automation Favors the Offense

Attackers have always benefited from scale and, unfortunately, AI simply amplifies that advantage.

Modern attack campaigns will utilize automation not to invent entirely new exploits, but to industrialize existing ones. Known vulnerabilities, misconfigurations, and weak identity controls are now stitched together by AI-assisted tooling that adapts quickly, probes relentlessly, and exploits opportunity at machine speed. These campaigns are quieter, more persistent, and harder to distinguish from background noise.

Critically, attackers do not need perfect precision. They benefit from volume, iteration, and probabilistic success. A small improvement in targeting or evasion, multiplied across thousands of attempts, yields meaningful results and AI excels at this kind of iterative optimization.

Why Defensive AI Struggles to Keep Up

On the defensive side, AI is often deployed as an enhancement to existing tools, offering faster detection, better prioritization, and smarter correlation within an enterprise SIEM, for example. These are valuable improvements, but they do not change the fundamental constraints under which defenders must operate.

Security teams are accountable for outcomes, needing to explain decisions, justify actions, and demonstrate control. False positives disrupt business. False negatives create risk. Every automated response must be defensible after the fact to management, auditors, regulators, or customers.

This asymmetry matters. An attacker can afford to be wrong repeatedly, trusting that eventually a shot will be on target. Defenders, just like the poor old goalkeeper in soccer, cannot afford to be wrong once.

AI-powered detection may surface more signals, but without clear governance, visibility, and control, those signals quickly become noise. Automation without accountability simply accelerates confusion.

The Persistence of “Old” Attack Surfaces

One of the more uncomfortable realities emerging from recent incidents is that many AI-enabled attacks still rely on very traditional weaknesses, such as exposed services, misconfigured cloud environments, weak access controls, unpatched software, and multiple evasion techniques. AI does not replace these attack vectors but instead makes them easier to discover and exploit at scale.

To any seasoned security professional this sounds very familiar. The old saying goes: “there is nothing new under the sun” and the danger is not that AI introduces entirely new classes of risk overnight (although it inevitably will – see our white papers “AI Security Beyond the Model: What Enterprises Need to Care About — and Why” and “Evaluating Enterprise AI Security: Questions Every Buyer Should Be Able to Answer”). The real danger in the short term is that it quietly magnifies existing weaknesses until they become systemic failures.

Why Symmetry Is the Wrong Mental Model

The idea of a balanced AI arms race assumes that both sides gain comparable benefits from automation. In reality, the incentives and constraints are fundamentally different.

Attackers optimize for opportunity and speed, while defenders optimize for stability, correctness, and trust. AI aligns naturally with the former, aligning with the latter only when paired with strong governance, observability, and control.

This is why simply “adding AI” to security tools does not meaningfully close the arms race gap. Without clear policies, auditability, and predictable behavior under stress, automation can undermine confidence rather than strengthen it.

Reframing the Defensive Objective

The goal of AI security should not be to match threat actors’ algorithm for algorithm, since that race is unwinnable and misses the point.

AI-enabled systems must be designed and evaluated not just for detection capability, but for how they behave when assumptions break, when inputs are ambiguous, dependencies fail, or automation makes the wrong decision.

This requires shifting emphasis away from novelty and toward discipline and focusing on visibility into how decisions are made, evidence that controls behave predictably under stress, and governance that aligns automation with enterprise risk tolerance.

What CISOs Should Do Now

  • Treat AI-enabled security controls as governed systems, not intelligent features. Demand clarity on how automation is authorized, constrained, and audited.
  • Insist on observability and accountability. If your team cannot reconstruct why an automated decision was made, you cannot defend it to regulators, boards, or customers.
  • Pressure-test failure modes. Ask how controls behave when dependencies degrade, data is ambiguous, or models behave unexpectedly.
  • Resist the temptation to equate speed with strength. Automation that outpaces governance increases risk rather than reducing it.

Enterprises that succeed in this environment will not be those that deploy the most AI, but those that govern it best.

The uncomfortable truth is that AI makes security failures more likely when organizations do not understand their own systems. The answer is not less automation, but better control over how automation is introduced, evaluated, and held accountable.

The arms race metaphor obscures this reality. Defense is not about symmetry, but responsibility and accountability. And in an AI-enabled world, responsibility is the hardest capability to automate.