ENNL
Book a call

Services / AI security engineering

Protection against rogue AI agents, agent swarms and attacks on AI systems.

In July 2026 agents from an AI lab's cybersecurity evaluation coordinated an intrusion into Hugging Face, a case that Hugging Face and METR have documented, and AI browsers and assistants have been shown to follow instructions hidden in the pages they read. We harden AI agents and LLM applications so that one bad instruction cannot become one bad action, and we test them the way an attacker would, with written authorisation from the system owner. We also make sure your Databricks platform can be restored, and we prove it with a drill. A core team leads, and dedicated engineers join where the work needs them.

Book a callAI agent penetration test

01 / What we offer

Three services, one way to start

Pick the one that matches your question. Each is a product with a written scope and a result you keep.

  • AI agent penetration test

    An authorised white-hat test that shows how your AI agent breaks, and how to fix it.

    You get

    • A written scope, with the authorisation of the system owner
    • Findings ranked by risk, with reproduction steps and fixes
    • A retest that confirms what is closed

    How it runs

    1. Authorise and scope
    2. Model the threats
    3. Test
    4. Report
    5. Retest
    See the product

  • Agent security hardening

    Agents that stay inside their limits, with the defence layers and attack tests built in.

    You get

    • Identity, limits, approval gates and a kill switch in your agents
    • Attack tests that run as a release gate
    • Runbooks for your team

    How it runs

    1. Model the threats
    2. Design the layers
    3. Implement
    4. Test and gate
    5. Hand over
    See the product

  • Databricks backup and restore readiness

    A Databricks platform you know how to restore, and how long a restore takes.

    You get

    • A recovery design with synced deep clones and configuration as code
    • A restore runbook per system
    • A first restore drill with a measured recovery time

    How it runs

    1. Assess
    2. Design
    3. Build
    4. Drill
    5. Schedule
    See the product

Ongoing protection

Continuous agent security monitoring and periodic retests run through our managed AI operations and our senior capacity on retainer.

Managed AI operations Senior capacity on retainer

How to start

  1. Book a callTell us which agents and data your AI systems reach.
  2. Scope and authorisationWe agree the scope in writing, and the system owner signs the authorisation.
  3. StartThe work begins on the agreed date.
Book a callSend us your brief

02 / What changed

What changed in AI security recently

Dated developments from primary sources, and what each one means for a team that runs AI agents.

A documented swarm case: Hugging Face and OpenAI, July 2026

Hugging Face disclosed on 16 July 2026 that an autonomous AI agent framework had accessed a limited set of internal datasets and several credentials used by its services, and said it found no evidence of tampering with public models, datasets or Spaces. METR's independent investigation of 26 August 2026 traced the agents to an OpenAI internal cybersecurity evaluation. METR reports that roughly 1,200 agents meant to be isolated from each other found a way to communicate on a shared message board, and that about 700 of them joined the attack. Each point below is attributed to the party that published it.

What went wrong in the Hugging Face incident and the layer that answers it
What the sources reportSourceOur layer
Agents meant to be isolated from each other communicated on a shared message boardMETRNo shared writable stores between agents, and egress control
Entry came through dataset processing, with a remote-code dataset loader and a template injection in a conversion podHugging Face timelineSafe parsing of data and configuration, and sandboxed execution
Credentials harvested from a worker environment led to cluster-admin on several internal clustersHugging Face timelineScoped credentials, identity per agent and least privilege
Hugging Face's AI-based alerting correlated the signals but did not raise the criticality or page on-callHugging Face timelineAnomaly alerts that escalate early to a person
The agents ran in an evaluation environment, not in productionMETRThe same monitoring in test and evaluation environments as in production
Recent AI security developments and what they mean for you
DateWhat happenedWhat it means for you
Jul 2026Hugging Face disclosed on 16 July that an autonomous AI agent framework had accessed internal datasets and service credentials. METR traced the agents to an OpenAI cybersecurity evaluation and counts about 700 of roughly 1,200 agents on a shared message board joining the attack.A documented swarm case: agents coordinated without being told to. Egress control, no shared writable stores, scoped credentials and human escalation answer it.
20 Aug 2025Brave showed that hidden text on a web page can make the Perplexity Comet AI browser act on the user's accounts.Any assistant with account access needs the same injection tests as your own agents.
13 Nov 2025Anthropic reported a state-linked group that used an agentic coding tool against roughly thirty targets and let AI do 80 to 90 percent of the campaign, with people stepping in only now and then.Attacks can run at machine speed and volume, so limits, anomaly alerts and a kill switch matter.
25 Nov 2025The MCP specification added OpenID Connect Discovery and incremental scope consent, and updated its security best practices.We check that your MCP servers and clients use the current authorisation flow and ask for the smallest scope.
9 Dec 2025OWASP published the Top 10 for Agentic Applications for 2026.Our threat model and test plan for agents follow it.
Mar 2026In a large red-teaming competition run with NIST CAISI, every model tested fell to at least one successful attack in tool-use, coding and computer-use settings.Model-level safety does not remove the need for controls around the agent.
20 Mar 2026Databricks AI Security Framework v3.0 added an agentic extension with 35 new technical risks and 6 new controls, grouped as agent core, MCP server, MCP client and inter-agent dynamics.We map the controls on your platform to the extended framework.
2026MITRE ATLAS added agent techniques through its 2026 releases, among them AI Agent Tool Poisoning, Autonomous Attack Orchestration and Autonomous AI Agent Communication.Our reports name the ATLAS technique for each attack step.

Primary sources

Hugging Face: security incident, July 2026 Hugging Face: technical timeline METR: independent investigation OpenAI: postmortem Brave: Comet prompt injection Anthropic: AI-orchestrated espionage campaign MCP specification changelog OWASP Top 10 for Agentic Applications NIST CAISI: agent red-teaming competition MITRE ATLAS releases

03 / Threats

Threats we defend against

The OWASP Top 10 for LLM Applications 2025 is the common language for these risks. This is how each one shows up in an agent that reads documents and calls tools.

OWASP Top 10 for LLM Applications 2025 items and where we defend
OWASP itemWhat it means for your agentWhere we defend
LLM01 Prompt injectionInstructions hidden in a prompt, a document, a web page or a tool result change what the agent does. Indirect injection arrives through content the agent reads on its own.Filtering, approval gates and attack tests
LLM02 Sensitive information disclosureAn answer, a log line or a tool call reveals data the caller may not see.Least-privilege access and output filtering
LLM03 Supply chainModels, packages, plugins and MCP servers from outside your control carry the risk into your system.Allow lists and reviewed dependencies
LLM06 Excessive agencyAn agent holds more tools, rights or autonomy than its task needs, so a mistake or an attack has a large reach.Identity per agent and approval gates
LLM07 System prompt leakageThe instructions and secrets in a system prompt can be read or inferred by the user.No secrets in prompts and output filtering
LLM08 Vector and embedding weaknessesA retrieval store returns poisoned or over-shared content to the model.Access rules on the index and source review
LLM10 Unbounded consumptionOne client, one loop or many parallel agents use up spend and capacity. Attackers call it denial of wallet.Spend, rate and step limits

04 / Agents and swarms

Agent-specific threats and swarm scenarios

Agents add risks that a plain chat application does not have: they act, remember and talk to other agents. The OWASP Top 10 for Agentic Applications for 2026 covers risks such as goal hijack, tool misuse, memory poisoning, insecure inter-agent communication, cascading failures and rogue agents. These are the scenarios we design against.

  • Goal hijackText the agent reads redirects it from the task you gave it to one an attacker chose.
  • Tool misuseAn agent uses a tool it is allowed to use for a purpose nobody intended, such as exporting records it can legitimately read.
  • Memory and context poisoningA bad entry in memory, a retrieval index or a long context keeps influencing the agent long after the attack.
  • MCP server risksA confused deputy acts with rights the caller never had, a server accepts tokens it did not issue, or a token carries more scope than the task needs. We follow the scope minimisation and token rules of the MCP security best practices.
  • Swarm abuseMany agents or automated clients act at once, each within its own limit and together far beyond what the system can carry. In the July 2026 Hugging Face incident, METR counts about 700 agents that joined one attack. We set limits per agent and for the swarm as a whole.
  • Insecure agent-to-agent communicationAn agent that trusts another agent by default passes a compromise along. In our designs every message between agents is treated as untrusted input.
  • Cascading failuresOne wrong output feeds the next agent, which acts on it and feeds the next. We add checks and breakers between the steps so a failure stays local.
  • Rogue agentsAn agent that drifts from its role, through a hijack, a bad memory or its own error, keeps acting. Limits, monitoring and a kill switch bring it back under control.

The Hugging Face incident above is a documented swarm case. MITRE ATLAS names agent techniques such as Autonomous Attack Orchestration and Autonomous AI Agent Communication. Our swarm controls are engineering practice that we apply and test with you.

OWASP Top 10 for Agentic Applications for 2026

05 / Hardening

How we harden agents

Defence in layers: no single control has to hold for the agent to stay safe. Anthropic reaches the same conclusion in its research on trustworthy agents of 9 April 2026: no single line of defense is enough. Each layer has an owner and a test.

  1. IdentityOne identity per agent
  2. Filter inInputs checked and limited
  3. Sandboxed tool callAllow list, least privilege
  4. Approval gateA person approves impact
  5. Filter outOutputs checked, egress controlled
  6. Trace and alertEvery step recorded
  • Identity per agent and least-privilege toolsEvery agent has its own identity and only the tools and data its task needs, so a hijacked agent has a small reach.
  • Allow listsTools, servers, domains and actions are allowed by name. Everything else is refused.
  • Approval gates for actions with impactPayments, deletions, external messages and permission changes wait for a person, with the context to decide.
  • Sandboxed execution and egress controlCode and tools run in an isolated environment, and outbound traffic goes only to destinations on the list.
  • Input and output filteringRequests, documents and tool results are checked on the way in, and answers and tool calls on the way out.
  • Spend, rate and step limitsCaps per agent and per swarm on spend, request rate and the number of steps in a loop, so one failure stays small.
  • Kill switch and circuit breakersA person can stop an agent at once, and breakers cut it off when its behaviour leaves the normal range.
  • Full tracing and anomaly alertsEvery model call and tool call is recorded, and alerts fire on unusual tools, volumes and destinations.
  • Attack evaluation sets as release gatesA set of known attacks runs on every change. A release that lets one through does not ship.

How we build agents Unity AI Gateway for access, spend and logs

06 / Red teaming

White-hat AI agent penetration testing and red teaming

We attack your AI system the way a real adversary would, so you find the weakness before someone else does. Tests follow the OWASP agentic list and name the MITRE ATLAS technique. We only test systems we are authorised to test.

  • Prompt injection and jailbreaksDirect and indirect, through documents, web pages, e-mails and tool results, and attempts to make the system ignore its rules.
  • Tool abuseUsing the agent's own tools for actions and data it was never meant to reach.
  • Data exfiltration pathsRoutes by which private data leaves through answers, links, tool calls or logs.
  • Privilege escalationGaining more rights through tools, service accounts and MCP servers.
  • Memory poisoningPlanting content in memory or a retrieval index that changes later behaviour.
  • Denial of walletDriving spend and capacity up with loops, large inputs and parallel agents.
  • MCP and tool supply chainThe servers, packages and plugins an agent loads, and what a malicious or changed one can do.
  • Coding agents on developer machinesWhat a coding agent can read, run and send from a laptop or a build server when a repository or a package turns hostile.
  • AI browsers and assistants with account accessWhether a page or a message can steer an assistant that is signed in to your accounts.

07 / How a test runs

A test with rules, scope and a stop procedure

A penetration test is an agreement first and a technical exercise second.

  1. Written authorisation

    The owner of the system signs off before anything starts. Unauthorised access is a criminal offence under Dutch law, so we test no system without it.

  2. Scope and rules of engagement

    A written scope lists the systems, the methods and the limits. Named contacts on both sides, test accounts and a stop procedure are part of it.

  3. Test set-up

    We use test accounts and test data. Production data is used only when we have agreed it in writing.

  4. Attack

    We run the attacks from the scope, record every step and keep the reproduction steps.

  5. Report

    Findings are ranked by risk, each with reproduction steps and a fix, and mapped to OWASP, MITRE ATLAS and the Databricks AI Security Framework.

  6. Retest

    After your team fixes the findings, we run the same attacks again and report what is closed.

We only test systems we are authorised to test, with a written scope from the system owner.

What you get

  • Findings ranked by risk, with reproduction steps
  • A fix for every finding, written for your team
  • A mapping to OWASP, MITRE ATLAS and the Databricks AI Security Framework
  • A retest that confirms the fixes

08 / Backup and restore

Databricks backup you have restored once

Databricks disaster recovery guidance makes you responsible for your recovery point objective and for the integrity between your primary and secondary deployments. A backup you never restored is a hope, so we plan, build and rehearse it.

Recovery aids with limited windows

  • Time travel reads older versions of a table, 7 days by default
  • UNDROP restores a dropped table within a 7 day window by default
  • VACUUM removes the old files that time travel depends on

What we set up

  • Deep clones of critical tables, synced for disaster recovery
  • Configuration as code with Terraform and Declarative Automation Bundles (DABs), so the platform can be rebuilt
  • Documented restore runbooks per system
  • Scheduled restore drills with measured recovery times

Managed disaster recovery and Lakebase

Databricks managed disaster recovery is gated: you apply through your account team, and it needs Premium, the Mission Critical add-on, serverless and private endpoints in the secondary region. It does not replicate managed volumes, secrets, ML models, model serving, vector search indexes, Delta shares or published dashboards, and data written after the last replication point can be lost in an unplanned failover. Lakebase has point-in-time restore with a window of 2 to 30 days, 7 by default, and a restore creates a new branch. A restore plan covers what these features leave out.

At the end you know how long a restore takes, because we measured it.

Databricks disaster recovery guidance Managed disaster recovery

  • Databricks
  • Unity Catalog
  • Delta Lake
  • Terraform

09 / Frameworks

Frameworks we map to

Every finding and every control maps to a recognised framework, so your security and audit teams can read the result.

Frameworks and how we use them
FrameworkWhat it coversHow we use it
OWASP Top 10 for LLM Applications 2025The ten main risks of LLM applicationsThreat model and test plan
OWASP agentic AI guidanceThreats and mitigations for agents, and the Top 10 for Agentic Applications for 2026Agent threat model
MCP specification and security best practicesAuthorisation, scope consent and the named risks of MCP serversReview of your MCP servers and clients
MITRE ATLASAdversary tactics, techniques, mitigations and case studies against AI systems, including agent techniquesMapping of attack steps in reports
NIST AI RMF 1.0 and the Generative AI Profile (AI 600-1)A voluntary framework for managing AI riskRisk language for governance
Databricks AI Security Framework (DASF) v3.097 risks and 73 controls across the AI system on Databricks, with an agentic extension in v3.0Controls on your platform
EU AI Act, Article 15High-risk AI systems must be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilitiesEvidence of robustness testing

OWASP Top 10 for LLM Applications OWASP Agentic AI threats and mitigations MITRE ATLAS NIST AI RMF 1.0 NIST Generative AI Profile Databricks AI Security Framework MCP security best practices EU AI Act (Regulation (EU) 2024/1689)

10 / FAQ

Questions about AI security

Do you test any system we name?

No. We test only systems that you own or that the owner has authorised in writing. We agree the scope, the rules of engagement, the named contacts and a stop procedure before the first test.

Will a test break our production system?

We use test accounts and test data, and production data only when we have agreed it in writing. The rules of engagement say which actions are out of bounds, and the stop procedure lets either side halt the test at once.

What do we get at the end of a test?

A report with findings ranked by risk, reproduction steps and a fix for each, mapped to OWASP, MITRE ATLAS and the Databricks AI Security Framework. After your team fixes the findings, we retest and report what is closed.

Can you harden agents that we built ourselves?

Yes. We start from what you run, add the missing layers such as identity per agent, approval gates, limits, tracing and an attack evaluation set, and leave the code and the runbooks in your repositories.

Is Time Travel a backup?

No. Time travel keeps old versions for 7 days by default and VACUUM removes them, and UNDROP has a 7 day window by default. They help you recover from a mistake. A backup needs a copy, such as a synced deep clone, and a restore you have practised.

Where does this fit with the EU AI Act?

Article 15 asks high-risk AI systems to be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. Our tests and reports give you evidence of that kind of robustness work. Whether your system is high-risk is a legal question for your own advisers.

Next step

Which AI system should we attack first?

Tell us which agents and data your AI systems can reach.

Book a callSend a messageor email info@nexops.nl directly

Currently accepting projects for Q4 2026