By now, many people have seen public reporting where an AI agent reportedly decided, for some reason, to delete its company’s database. To be very clear: SRA has no inside information on that event and is simply reading the news like everyone else. But even from the outside, it feels like a useful moment to highlight a small number of software architecture choices that may have helped prevent that sort of thing from happening.

In cybersecurity, we need to be proactive in understanding how AI-enabled software actually works. More importantly, we need to be actively engaged with software development teams and not just when it’s time to test what they built, but during the design phase itself. That urgency is driven by the overwhelming pressure to get AI-enabled software into production as quickly as possible. At the same time, there is basically no universally agreed-upon set of secure design patterns for agentic software today. There are plenty of good practices. But each agent’s design should be fully understood and re-evaluated every time new skills, new capabilities, or new usage modalities are added.

A very common agent design pattern today is ReAct, short for REasoning and ACTion.  The Reasoning part is the “brain,” usually a language model. It interprets user input, plans what to do, decides what tools to call, and synthesizes an answer.  The Action part is how the agent interacts with the outside world: retrieving data, calling APIs, querying databases, writing files, sending messages, and so on.  Often, the Reasoning portion is considered non-deterministic, while the Action portion should be considered deterministic. Those are fancy words for unpredictable and predictable.

LLMs are non-deterministic in the sense that the same input will often produce somewhat different output each time; a lot like a human. That is not inherently bad. It is often the whole point. But when it comes to actions taken against the outside world, we should be highly deterministic to enforce limits on an agent’s boundaries and capabilities. Think of that deterministic tool layer as a security buffer between your unpredictable LLM and, say, your entire company’s databases.  Or, to draw from the endless wisdom of Dr. Egon Spengler in Ghostbusters: “Don’t cross the streams.”  In this context, that means don’t mix your non-deterministic reasoning with unrestricted deterministic execution. If you do, you are setting yourself up for total protonic reversal, all life as you know it to stop instantaneously, and every molecule in your body to explode at the speed of light. Or, in less scientific terms: your company database gets nuked.

A hyper-simple example

Let’s tee up a very simple example: an agent that takes user input, gathers data from a database, and then generates an answer to a question.

That is a hyper-simple example, but honestly, this is how the sausage is made a lot of the time. We created an agent, gave it a model, and assigned it one tool. That tool — the `database_access` function — is eventually going to become our interface to the company database.

Before we change anything, it is important to understand the tool. It is a reusable function that the agent can call, and the agent can send it one message called `inputValue`. That is literally all the agent can do with it. There is no other path or mechanism for the agent to interact with the outside world.

In this case, the agent effectively does the following:

1. Receives input from the user.
2. Considers whether it can solve the user’s question with the data it currently has, or if it needs to use one of its tools. It is aware of the tool by interpreting the function’s docstring — `”””Look up information in the database based on the user’s request.”””` — as context for what skill it has access to.
3. Chooses to execute the selected tool by looking at the input options it is allowed to send to the tool. In this case, that is just `inputValue` and its description.
4. The tool executes and returns data.
5. The agent effectively goes back to step 2 and reconsiders whether it can now answer the user, or whether it needs to call a tool again.

So far, so good.

Now let’s talk about tool design, and how to absolutely cross the streams

Below is a terrible implementation of our tool.  Why is it terrible?

Because it allows the LLM to generate the exact SQL query to run against the database, and then it executes that query unchecked. In other words, you took your tool’s access to the outside world and made it completely controlled by the AI, then allowed it to execute blindly.

No amount of system prompt wording, careful instruction, or pleading will ever guarantee good behavior in a non-deterministic compute environment.

This exact pattern will happily delete your entire database if the user asks and the model decides to comply.

And if your immediate thought is, “Yeah, but I told it in the system prompt to never take destructive actions or run delete commands.” That is not a control. That is a suggestion. Your LLM should be treated like a two-year-old. It might have good intentions. It might even understand the instructions. It is still going to run into the parking lot sometimes.

The problem here is not just “the AI behaved badly.” The deeper problem is architectural:

• The LLM is generating executable instructions.
• The tool blindly trusts those instructions.
• The tool has destructive capability.
• There is no deterministic boundary between “reasoning” and “doing.”

That is crossing the streams.

A better pattern: narrow, deterministic, read-only tools

Here we can apply concepts security practitioners have known for a very long time – including lessons from the OWASP Top 10 2010 era around input handling, argument parameterization, allowlists, and constrained execution. Instead of allowing the LLM to write raw SQL directly, we give it a safe set of options:

• a list of tables to choose from
• a limited list of safe columns
• a bounded number of results
• safe filter operators
• parameterized filter values

Notice how the class that defines the tool input changes. That schema dictates the options given to the AI. There is no existing code path whatsoever that could perform a destructive action.

So even if the model *wants* to do something destructive, the tool literally has no destructive capability to invoke.

That is what “don’t cross the streams” looks like in practice.

The real lesson

To be clear again, we do not have any inside information on the real-world event that prompted this discussion. We are not claiming this is what happened, but this is what we see every day. We are simply pointing out that when an agent has access to real systems, software architecture choices matter enormously.

This is not the only control layer that should exist. Far from it. Mature agentic systems should also be thinking about:

• least privilege on underlying credentials
• read-only database accounts where appropriate
• approval workflows or supervisors for sensitive actions
• output filtering and policy enforcement
• audit logging and traceability for tool use
• environment separation
• rate limiting
• monitoring and alerting
• human-in-the-loop controls for high-risk actions
• kill switches and rollback planning
• secure code review and threat modeling

But the tool boundary is still one of the most critical places to get right, because it is one of the most likely places for errors to become amplified into incidents.

The bad example above is intentionally extreme for teaching purposes. Most real environments are not quite that reckless. But, the core lesson is very real: we should be focusing right now on secure software design architecture, code reviews, penetration testing of agentic behavior, and close partnership with developers during this period of rapid change and innovation.

SRA offers a range of services to support clients throughout this journey, from Red Team engagements to standard assessments. We also perform architectural reviews of corporate and application-level AI implementations. Key services include:

• Security Architecture – Ensure your implementation is built on the right foundation
• Code Review – Identify complex logic flaws and receive detailed guidance
• Penetration Testing – Validate that no vulnerabilities are introduced along the way
• AI Developer Enablement – Equip your developers with the tools they need

Whether you’re establishing AI safety practices from the ground up or bringing uncontrolled AI use under control, SRA is the trusted partner to guide you every step of the way. So, remember, don’t let your business’ FOMO let your developers code like YOLO.