The Asymptote of the AI SOC is Determinism

As a security engineer, I can't avoid the hype around "AI SOC." The industry seems too willing to give an agent a bunch of tools before asking whether the problem could be solved deterministically. I think AI will keep improving and meaningfully improve the SOC, but much of that value will come from making deterministic automation cheaper to build. As that happens, I expect repeatable work that now goes through agents to move into deterministic code.

Right now, AI looks to me like Maslow's hammer. We have this cool new technology, so we need to use it wherever possible. However, many AI initiatives fail to offer a strong return on investment [1]. A meaningful AI strategy needs to consider both how AI can expand a team's capabilities and when traditional engineering may be a better fit.

Runtime agents bring real benefits, but they come with meaningful trade-offs. I am particularly worried about marginal compute costs and the unreliability that comes with nondeterminism.

Automation and Marginal Costs

Before discussing how AI may impact automation, we need to understand why we automate. One of the most formative books I have ever read is about the mass manufacturing of cars. I see three characteristics as critical to automation:

1. Automation typically exchanges higher up-front costs for lower variable operating costs. Because this fixed cost can be distributed across repeated executions or units of production, automation becomes more economically attractive as volume increases [2].
2. Automation is generally most attractive for processes that are frequent, stable, standardized, and governed by well-defined rules. Processes with substantial variation or numerous exceptions need more implementation effort because the automated system must recognize and handle those exceptions [3].
3. Engineers therefore have a strong incentive to reduce process variability when the cost is justified. As the number of independent dimensions of variation increases, the potential configuration space grows combinatorially [4]. This correlates with an increased engineering workload, even if the workload does not necessarily grow combinatorially [5].

Automation is highly sensitive to marginal inefficiencies and costs. The biggest barrier to automation is the high capital investment. Resources are limited, engineers are expensive, and security investments usually do not have a causal link to revenue.

Most AI SOC vendors frame their agentic systems as replacing manual work. Agents compare favorably to full-time employees on marginal costs (per-case resource usage) and elasticity (adjusting to alert volumes). In this way, agents are a good middle ground between people and deterministic automation. The question is how much of that work will still need an agent as deterministic automation gets cheaper to build.

AI inference is expensive, especially for frontier models. Even if it commoditizes, efficiency will still determine costs. LLM inference requires substantial compute and memory, even for tasks that a small amount of deterministic code could handle. For repeatable tasks, a GPU-backed AI agent is unlikely to match the marginal cost of CPU-backed automation.

Capital Costs are Falling

Alongside runtime capabilities, AI also drastically reduces the capital investment needed for new automations. AI coding tools are powerful, and a smart engineer working with a capable AI model can achieve far more per hour than before; for most security teams, the biggest engineering cost center is work-hours.

The biggest mistake in betting heavily on AI agents is assuming the cost of determinism will stay roughly where it was before. Processes that weren't worth automating deterministically can suddenly deliver meaningful ROI, and engineering teams can work through their backlog much faster. While agent marginal costs eat away at the share of work done manually, AI-assisted programming eats away at the work that agents might do.

The important question is whether a workflow is worth automating over its useful life. That includes building, validating, and maintaining it, alongside the cost of each run. AI-assisted programming lowers the threshold for deterministic automation to pay off, even as inference gets cheaper.

Nondeterminism Brings Unpredictability

After an initial prompt is submitted, AI systems can branch off in wildly different directions. Tool calls and retrieval-augmented generation (RAG) can help keep a system grounded, but they do not guarantee repeatable behavior. AI systems still rely heavily on prompt engineering and are difficult to control.

Agents can help handle unmodeled edge cases, but unpredictable execution is really bad for achieving repeatable, trusted behavior. Deterministic code can still be wrong, but repeatable behavior makes it easier to test, debug, and audit. Security Operations overwhelmingly consists of high-frequency hot paths, which is why SOAR exists. An autonomic SOC can usually be modeled as a high-level decision system (deterministic, agentic, or human-led) interacting with well-defined, repeatable procedures. Examples include searching browser history for malicious sites or blocking an indicator on a firewall. Introducing variability to these processes is more likely to make outcomes worse, not better.

Sales demonstrations for agentic SOC products often rely on workflows that could be done deterministically because they are easy to demo and raise fewer questions about trusting the AI to act autonomously. These demos are then prompt-engineered to hell and back. During proofs of concept for agentic SOC products, I've often found myself spending more time prompt-engineering an agent than it would take to build the logic deterministically.

LLMs can interpret language and generalize beyond explicit rules. That makes them useful for complex situations and cases where there isn't one right output. Those capabilities are valuable even when we want predictable execution. Still, most of the work we do in D&R engineering can be modeled deterministically.

Determinism vs AI or Determinism + AI

I think determinism and AI will become deeply integrated, but determinism will maintain the primary role, at least on the engineering side. Engineers can move faster and achieve greater coverage by working with an AI editor than by manually building playbooks. These playbooks retain the cost and trust benefits of deterministic systems.

SOAR playbooks are particularly suited to AI-assisted deterministic automation because their logic is often straightforward. For the most part, it is a matter of API coordination and high-level control flow. SOAR deployments also tend to have well-defined subplaybooks with clear input and output schemas, which makes it easier for AI to write calls with the proper signature.

In my own detection engineering experiment, I started with an agent-heavy workflow. As I developed the harness, I found parts that were better handled deterministically. That also let me use smaller models with results I was happy with.

Some of the clearest uses I've seen for runtime agents include:

1. Normalizing unstructured data to a standard schema (optionally including other tool calls), for example, converting plain-language threat intel reports into a machine-readable document.
2. Automating low-frequency tasks or edge cases that are not worth modeling deterministically, such as searches for infrequent indicator types in a SIEM.
3. Achieving an end goal when the intermediate steps are too complex to model within a SOAR's control flow capabilities, like recursively unwrapping a ClickFix attack
4. Processing/writing natural language or code, for example, automatically writing detection rules based on a set of criteria

All of these are cases where LLMs' properties are useful. For the most part, they augment deterministic systems.

I'm Just Jev

After I had finished writing this post, my X feed blew up with discussions about a new model called Jev. Jev is a new classification model that works really well with abstract data without much custom fine-tuning. It quickly became the fastest model used ever on the Vercel AI Gateway.

I don't think short-term hype cycles are a productive way to drive long-term adoption, and I think much of the online discourse comes from people who don't really understand what Jev is useful for (no, your "Jarvis" isn't going to run entirely on Jev). Also, a lot of the demos I've seen online don't use Jev within its capabilities, so take the online discussion with a mountain of salt.

At the same time, Jev is the first model I've seen in a long time that I really could see being useful in the future. Classification algorithms are great at turning unstructured data into structured data, which is the most compelling use case I see for AI. Jev is also much cheaper than LLMs (for the tasks it supports), which makes it far more deployable than the equivalent agent. There are tons of cases, especially classification in triage (for example, trying to see if an email is phishing), where this could be a really useful step in a deterministic system.

While I am critical of some of the online discussion around Jev, the hype is also grounded in something real. There is huge demand for a model that is easy to work with, but instead of trying to be an AGI, it tries to solve a more limited problem well. Hopefully the hype around Jev encourages more development in these areas with very clear ROI.

Is Your SOC "Deterministic Ready?"

This brings me to one of my biggest concerns for Detection and Response right now: the world seems to be moving towards agentic capabilities at the expense of deterministic capabilities. Both established SOAR vendors and startups seem united in the view that the future is an agentic SOC. In my discussions with vendors, I see a strong correlation between selling this vision and having half-baked deterministic engines.

SOAR is a difficult product to do well. When designing a platform, engineers make early architectural decisions with lasting trade-offs. Once the product ships, changing those foundations can force customers to rewrite their automations. The decisions that worry me most are:

• Inefficient runtimes. For SOAR, it is very difficult to find a runtime environment that is both efficient and flexible. Container-based platforms can bring long cold starts and high resource usage. For SIEM, schema-on-read hurts compression ratios and makes queries less efficient than columnar datatypes. Both of these are well-understood weaknesses, but are still in wide deployment.
• Poor control flow. Deterministic automation solutions need flexible control flow, with looping, parallelism, error handling, and reusability. Many SOAR platforms offer simplified control flow, which is great for getting started. However, it quickly turns automation into an exercise in bludgeoning workflows to meet platform limitations, instead of using the platform to build the most obvious workflow.
• Weak typing. I have another post in the works about this, but I am a strong believer that any security tooling needs a highly capable and easy-to-use type system. Nearly all SOARs have an underpowered type system, because it is hard to get right and often a low priority. If your SOAR platform relies on loose JSON blobs instead of strict schemas, your AI coding assistant will constantly hallucinate incorrect field names, breaking the automation at runtime. I tried one major SOAR vendor who relies heavily on JSON strings, but whose JSON handling was so bad it was easier to convert JSON to XML, process it, and then convert it back to JSON.

Some SOAR vendors have strong deterministic engines, but they are few and far between. I've seen legacy vendors make major architectural changes (essentially new products) for the "AI SOC" world, only to repeat the same architectural problems they had with their legacy products. And I am not confident many neo-SOAR startups have thought seriously about their deterministic engines or the trade-offs they made.

Strong deterministic capabilities will be a huge competitive advantage. I expect customers to pay for poor foundations through higher runtime costs, brittle playbooks, and painful migrations. As coverage increases, more traffic will hit SIEMs and SOARs, making underlying performance even more critical. Agents on bad fundamentals will still perform poorly.

As x Increases, y Approaches Determinism

AI has been transformational for me, and I am excited to see where the technology goes.

I use the word "asymptote" very intentionally. Many teams can benefit greatly from AI agents to get fast coverage, even if those agents are a transitional step. Regardless of cost pressure, AI brings genuinely novel capabilities that deterministic systems can't replace. I don't think any mature organization will ever end up back at 100% deterministic. However, as more of an agent's work becomes repeatable, though, I expect those steps to move into deterministic code. As such, both vendors and security teams should invest in deterministic capabilities and make sure that any "AI native" products adopted have robust technical foundations that don't turn AI into a long-term liability.

All opinions are my own and do not represent the views of my employer or any other organization that I may be associated with.