Introducing OASB: The Security Benchmark for AI Agents

OpenA2A Team

February 10, 2026

#oasb#security#benchmark#ai-agents#compliance

We created OASB (OpenA2A Security Benchmark) — an open security benchmark for AI agents. 46 controls across 10 categories with L1/L2/L3 maturity levels. Think of it as CIS Benchmarks for the agentic era.

# Run OASB-1 benchmark compliance check
$ npx hackmyagent secure --benchmark oasb-1

# Higher maturity levels
$ npx hackmyagent secure -b oasb-1 -l L2
$ npx hackmyagent secure -b oasb-1 -l L3

Spec at oasb.ai. Scanner at npm.

The Problem

AI agents are shipping to production without security standards.

When you deploy a web application, you have OWASP Top 10. When you configure a Linux server, you have CIS Benchmarks. When you set up AWS infrastructure, you have AWS Security Best Practices.

When you deploy an AI agent? Nothing.

This is a problem. AI agents can execute arbitrary code, access filesystems and databases, make HTTP requests to external services, read and write credentials, and interact with other agents. The attack surface is massive.

"The billion dollar question right now is whether we can figure out how to build a safe version of this system."

— Simon Willison

We think the answer starts with defining what "safe" actually means.

What is OASB?

OASB (OpenA2A Security Benchmark) is an open specification that defines security controls for AI agents. OASB-1 covers agent configuration security with:

46 controls across 10 categories
3 maturity levels: L1 (Essential), L2 (Standard), L3 (Hardened)
Automated verification via HackMyAgent CLI
Remediation guidance for each control

The benchmark is open source (Apache-2.0), vendor neutral, automatable, and mappable to SOC 2, ISO 27001, and NIST CSF.

The 10 Categories

#	Category	What It Covers
1	Identity & Provenance	Agent identity verification, cryptographic signing
2	Capability & Authorization	Permission boundaries, least privilege
3	Input Security	Prompt injection protection, input validation
4	Output Security	Response sanitization, data leakage prevention
5	Credential Protection	Secret management, credential rotation
6	Supply Chain Integrity	Dependency verification, package signing
7	Agent-to-Agent Security	Inter-agent authentication, communication logging
8	Memory & Context Integrity	Context injection protection, memory isolation
9	Operational Security	Resource limits, sandboxing, process isolation
10	Monitoring & Response	Security logging, alerting, incident response

Maturity Levels

Not every deployment needs maximum security. OASB defines three levels:

L1 - Essential

Baseline security every agent should implement. Covers the most critical risks with minimal overhead. If you're deploying any agent to production, start here.

L2 - Standard

Defense-in-depth for production systems. Adds controls for monitoring, supply chain integrity, and enhanced access control. Recommended for business-critical agents.

L3 - Hardened

Maximum security for high-risk or regulated environments. Includes cryptographic verification, formal audit trails, and advanced isolation. Required for agents handling PII, financial data, or operating in regulated industries.

Running OASB Compliance Checks

We built HackMyAgent to automate OASB verification:

# Install
npm install -g hackmyagent

# Run L1 benchmark
hackmyagent secure --benchmark oasb-1

# Run L2 benchmark
hackmyagent secure --benchmark oasb-1 --level L2

# Output formats for CI/CD
hackmyagent secure -b oasb-1 -f json
hackmyagent secure -b oasb-1 -f sarif -o results.sarif
hackmyagent secure -b oasb-1 -f html -o report.html

Example output:

OASB-1: AI Agent Security Benchmark v1.0.0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Level: Level 1 - Essential
Rating: Passing
Compliance: 85% (12/14 verified controls)

Categories:
  Identity & Provenance: 2/2 (100%)
  Capability & Authorization: 2/2 (100%)
  WARN Input Security: 2/3 (67%)
     FAIL 3.1: Prompt Injection Protection
  Output Security: 1/1 (100%)
  Credential Protection: 2/2 (100%)
  WARN Supply Chain Integrity: 1/2 (50%)
     FAIL 6.4: Dependency Vulnerability Scanning
  Operational Security: 2/2 (100%)

CI/CD Integration

Add OASB checks to your pipeline:

# .github/workflows/security.yml
name: OASB Compliance
on: [push, pull_request]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Run OASB-1 L1 Benchmark
        run: npx hackmyagent secure -b oasb-1 --fail-below 80

      - name: Upload SARIF to GitHub Security
        run: npx hackmyagent secure -b oasb-1 -f sarif -o oasb.sarif

      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: oasb.sarif

Compliance Mapping

OASB controls map to existing compliance frameworks:

OASB Control	SOC 2	ISO 27001	NIST CSF
1.1 Agent Identity	CC6.1	A.9.2.3	PR.AC-1
3.1 Prompt Injection	CC6.6	A.14.2.5	PR.DS-5
5.3 No Hardcoded Creds	CC6.1	A.9.4.3	PR.AC-1
10.1 Security Logging	CC7.2	A.12.4.1	DE.CM-1