Source: https://cortega.ai/blog/ailuminate-guardrails-vs-safety-models

# How General-Purpose Guardrails Compare With the Safety Models on the AILuminate List

Cortega team · November 2026

**TL;DR**

- No check flagged every hazard prompt. Cortega's strict policy check flagged 84%, Nemotron 3.5 Content Safety 82% and Llama Guard 4 58%.
- On attacks that build over a conversation, the two safety models caught 73% and 87%. Cortega's conversation check caught all of them in our own test set.
- The safety models are not in the AWS Bedrock catalog for us-east-1, so a customer hosts them on a GPU or calls a third party.

MLCommons publishes AILuminate, a public set of prompts built to push an AI system toward harm. It covers 12 hazard areas, from violent crimes to privacy. Its results page lists several safety models at the top of its ranking. We ran the public 1,200-prompt demo set through those models and through the guardrails we already ship.

## Why we ran it

Safety models are small classifiers that read one message and return safe or unsafe. A customer who adds one has a second model to host or call, and a check that sees one message at a time.

Most customers already run a guardrail for attacks that build up over a conversation, such as step-by-step escalation and long packed prompts. We wanted to know whether that same kind of guardrail, given a written policy, also covers the hazard areas.

## Result 1: no check catches everything

The results page ranks systems. It does not publish a catch rate. We measured one: the share of the 1,200 prompts each check flagged before any model answered.

| Check | Hazard prompts flagged | Harmless prompts flagged (37) |
|---|---|---|
| Cortega policy check on requests, strict setting | 84% | 1 |
| Nemotron 3.5 Content Safety | 82% | 2 |
| Cortega policy check on requests, standard setting | 72% | 1 |
| Llama Guard 4 | 58% | 0 |
| Cortega check, default settings | 46% | 0 |

## Result 2: attacks that build up

Hazard lists test single messages. Real attacks often start with a harmless question and add a little more each turn, or pack a long run of examples into one request. We ran our own set of single-message jailbreaks, many-shot prompts and multi-turn escalations through the same checks.

| Check | Single-message jailbreaks | Many-shot | Escalating conversations caught |
|---|---|---|---|
| Cortega conversation check, default settings | 100% | 100% | 100% |
| Cortega policy check on requests, standard setting | 74% | 100% | 100% |
| Nemotron 3.5 Content Safety | 94% | 100% | 87% |
| Llama Guard 4 | 89% | 100% | 73% |

This set leans toward credential and cyber attacks.

## Where the safety models run

Llama Guard 4 and Nemotron are open-weight models. Cortega Agentic Firewall uses models in the AWS Bedrock model catalog for us-east-1. A customer on that cloud has to host the Llama Guard 4 and Nemotron safety models on a GPU or call a third party (very high latency).

Not affiliated with or endorsed by MLCommons, Meta or NVIDIA.