AIShield Guardian – A Secure GenAI Adoption for Enterprises

While GenAI promises significant economic value, majority of IT leaders view it as a major cybersecurity risk, with fewer than 40% having established mitigation strategies. As part of AIShield's comprehensive AI Security solution development, this paper represents the first in a planned series dedicated to continuously evaluating our detectors and internal solutions against state-of-the-art industry benchmarks, with this inaugural version specifically addressing jailbreak detection capabilities.

This paper focuses on "jailbreak" attacks—adversarial prompts that bypass AI safety mechanisms—and demonstrates through comprehensive benchmarking that AIShield Guardian significantly outperforms competing enterprise guardrail solutions. Testing across independent benches such as WildGuard Bench and Jailbreak Bench datasets revealed that even advanced models like GPT-5 remain vulnerable to sophisticated attacks due to internal routing flaws, further outlining the need for external guardrails. AIShield Guardian provides a runtime-middleware defense framework that applies input and output filtering, policy compliance enforcement, intellectual property protection, and PII leak prevention. The jailbreak benchmarking results shows AIShield Guardian achieving superior accuracy (86.6% on WildGuard, 94.3% on Jailbreak Bench) while maintaining low false positive rates, positioning it as the most reliable enterprise-ready solution for secure GenAI adoption.

Harnessing the power of Generative AI (GenAI), enterprises are unlocking new levels of automation, creativity, and operational efficiency. Technologies such as Large Language Models (LLMs) are now deeply embedded across workflows from content generation and code synthesis to customer support, data analysis, and decision-making augmentation.

Despite this rapid adoption, security and compliance readiness continue to fall behind. A recent analysis estimates that GenAI could contribute between $2.6 and $4.4 trillion[1] to the global economy each year, yet more than 70 percent of IT leaders already classify GenAI as a significant cybersecurity risk. At the same time, fewer than 40% report having established and actionable mitigation strategies, revealing a substantial preparedness gap as organizations scale these technologies.

However, this rapid adoption brings a dual challenge: GenAI not only accelerates productivity and scalability but also introduces a new, less-understood attack surface. As these systems become widely adopted, they expose organizations to novel security, governance, and compliance risks that traditional controls are not designed to handle.

LLMs operate through probabilistic inference, generating responses by identifying and modeling patterns within vast training corpora. These probabilistic behaviors make them inherently vulnerable to adversarial inputs, a carefully crafted prompts, such as prompt injection, jailbreak attempts that can interfere with the model’s intended reasoning process and alter its outputs. Thus, systems can also produce inaccurate or misleading information, which poses significant risks to enterprises that depend on trustworthy and verifiable results.

As GenAI integrates into core business applications through APIs, automated agents, and enterprise specific models, organizations must reassess their existing security frameworks. Traditional controls that rely on static rules or perimeter defense are not adequate for systems that behave interactively and adapt to context. Effective protection requires continuous observation of model behavior, real time policy enforcement, that can respond to unexpected or evolving patterns in the model’s outputs.

Key concerns include:

• Lower technical barriers for threat actors: Adversaries can exploit LLMs to write malware, generate phishing content, or engineer social attacks.[2]
  
• Data leakage: AI responses can inadvertently expose proprietary data if trained on sensitive corpora, connected to Retrieval-Augmented Generation (RAG) or when user input/output handling is insecure.
  
• Unverifiable outputs: Hallucinations and ambiguous source attribution create challenges for compliance and auditability.
  
• Model misuse and over trust: Overreliance on LLM-generated suggestions in high-stakes environments (legal, medical, financial) introduces new systemic risks.

To responsibly adopt GenAI, enterprises must adapt to security architectures, data governance practices, and risk modeling strategies. This includes hardening interfaces, monitoring real-time prompt interactions, vetting model behavior under adversarial conditions, and aligning AI development with secure DevOps practices (SecDevOps). Without this transformation, organizations risk exposing themselves to emergent, AI-native threats that traditional tools are ill-equipped to handle.

The security posture of GenAI systems is shaped by attack surfaces that differ significantly from traditional software environments. The OWASP GenAI Security Project [3] highlights critical risks including prompt injection, insecure output handling, data poisoning, inference-time denial of service, insecure plugin design, model exfiltration, and uncontrolled model autonomy. OWASP’s Top 10 for LLMs [4] further emphasizes Prompt Injection and Insecure Output Handling as the most impactful categories, illustrating how untrusted user prompts and insufficient output validation can directly compromise system behavior. Complementing this, the MITRE ATLAS [5] framework provides an attacker-centric taxonomy of over 80 documented AI-specific techniques mapped across reconnaissance, resource preparation, exploitation, and post-compromise phases, enabling defenders to frame GenAI threats using structured patterns similar to how MITRE ATT&CK informs traditional cybersecurity strategy. Recent guidance from NIST [6] reinforces the severity of these risks, identifying prompt injection as a systemic vulnerability inherent to LLM design and noting that even well-aligned and well-governed models can be manipulated through indirect or hidden prompts embedded in text, code, or external data sources. The NIST report [6] stresses that current guardrails are insufficient, and that organizations must implement layered controls, contextual validation, and continuous monitoring to reduce prompt-based attack exposure.

For enterprise deployments, these risks can be categorized across three primary vectors Safety, Security, and Privacy. An overview of how each vector manifests within the GenAI threat taxonomy is provided below.

1. Safety Threats

Safety issues occur when the model produces harmful, biased, or non-compliant content due to its internal reasoning or training limitations, independent of any external attack.

• Toxic Outputs: Generating hate speech, misinformation, or violent content.
• Bias and Fairness: Reinforcing racial, gender, or cultural stereotypes.
• Overreliance: Users trusting hallucinated outputs without human verification.
• Unsafe Autonomy: Models taking unintended or unsafe autonomous actions when linked to downstream systems (e.g., code generation or decision engines).

2. Security Threats

Security risks involve deliberate adversarial actions targeting the model, its inputs, or the surrounding infrastructure with the goal of bypassing protections or compromising system integrity. Examples: -

• Prompt Injection/ Jailbreak Attacks: Crafting adversarial prompts to override system instructions.
• Model Evasion: Manipulating inputs to bypass filters or surveillance.
• Model Theft: Extracting proprietary model weights or intellectual property.
• Data Poisoning: Introducing manipulated data into training or fine-tuning. [7]
• Denial of Service: Overloading the model with resource-intensive requests.
• Supply Chain Attacks: Targeting pre-trained components or APIs integrated into the LLM system.

3. Privacy Threats

Privacy risks concern exposure, leakage, or inference of sensitive or regulated information through the model’s outputs or interactions

• PII Leakage: Model output includes personal identifiers (emails, SSNs, etc.).
• Training Data Memorization: The model regurgitates parts of proprietary datasets used in training.
• Membership Inference Attacks: Attackers determine if specific data was part of training.
• Cross-session Data Leakage: Information leakage between users in multi-tenant deployments.

This paper specifically focuses on the Jailbreak (JB) from the Security threat category. Our goal is to analyze and help enterprises defend against jailbreak attacks on GenAI systems. We evaluate attacks that surface in the real world and validate how AIShield Guardian can defend against such attacks and how well they prevent models from being coerced into unsafe outputs. We emphasize jailbreak because it directly challenges an AI’s safety mechanisms and has immediate implications for enterprise confidentiality and compliance as it bypasses the internal safety reasoning capabilities of the LLM.

Jailbreak Attacks

A jailbreak is an adversarial prompt or strategy that causes a model’s built-in safety evaluations to fail. In practice, attackers craft inputs (often by roleplay or encoded instructions) to convince the model to ignore its safety filters. Microsoft security researchers define an AI jailbreak as any technique that triggers a model to violate its guardrails, causing “the resulting harm [to come] from whatever guardrail was circumvented” [8]. In other words, a jailbreak bypasses the model’s intended restrictions. IBM analysts similarly describe jailbreaking as exploiting AI vulnerabilities to “bypass [ethical] guidelines and perform restricted actions” [9]. Common jailbreak techniques include injecting deceptive suffixes (such as instructions to disregard previous rules), persona-based manipulation (for example, asking the model to act as a malicious agent), multi-turn conditioning, and adversarial crafted prompts that disguise harmful intent. When successful, jailbreaks can cause a model to produce disallowed content ranging from operational security bypasses and unintended instructions to the leakage of proprietary or regulated information despite the presence of safety controls.

This paper prioritizes jailbreak analysis as the first area of focus, laying the groundwork for subsequent work that will examine other categories across the GenAI threat taxonomy covering the full spectrum of safety, security, and privacy risk vectors, supported by consistent benchmarking frameworks.

Both Jailbreak Benchmark and WildGuard Bench are recognized across industry evaluations, open-source, and academic, and are recognized as reliable benchmarks for assessing jailbreak resilience and harmful-content moderation. Their adoption in multiple research studies and public AI safety analyses positions them as trusted references within the AI security community.

We evaluate by measuring the jailbreak success rate: the percentage of adversarial inputs that still produce disallowed outputs. Lower success rates mean better protection. We compare AIShield Guardian against enterprise guardrail solutions: Google’s content safety API (Google Armor), Microsoft Azure’s Content Safety, AWS Bedrock Guardrails, OpenGuardrail.

We used GPT-5 a “judge” in our evaluation framework due to its advanced moderation capabilities and contextual understanding. Acting as an independent model, it systematically assesses whether outputs from each system violate safety policies or are susceptible to jailbreaks.

• Google Model Armor: Google’s new safety API “Guardrails” analyzes inputs/outputs for harmful content [12]. It is designed to flag or block policy violations before content is returned.
• Azure AI Content Safety: Azure AI Service provides Prompt Shields (formerly “jailbreak detection”) that catch prompt-injection attempts [13]. The Prompt Shield blocks or modifies user inputs deemed likely to provoke unsafe responses.
• AWS Bedrock Guardrails: Amazon Bedrock’s Guardrails feature offers configurable filters (content filter categories, denied topics, word/sensitive info filters, grounding checks) to catch disallowed requests or content [14].
• GPT-5 (as a Judge): GPT-5 is evaluated in a judge configuration to determine how effectively its internal safety mechanisms detect jailbreak attempts and classify policy-violating outputs.
• OpenGuardrails: An open-source framework that provides modular guardrails for LLM applications, enabling configurable input and output validation policies [15]. It supports checks for prompt injection, sensitive content, model-steering attempts, and harmful outputs, offering a transparent and customizable alternative to proprietary safety layers for enterprise GenAI pipelines.

Each Solution is evaluated under the same WildGuard Bench and Jailbreak Bench test suites to ensure a consistent and comparable assessment.

The results in Table 1 and Figure 1 summarize performance across both benchmarks using Accuracy, F1 Score, False Positive Rate (FPR) and False Negative Rate (FNR). This cross-benchmark view highlights not only how well each guardrail performs under general safety moderation, but also how reliably it maintains safety boundaries when subjected to deliberate adversarial manipulation.

The evaluation results across both benchmark suites demonstrate that AIShield Guardian provides the most reliable and consistent protection among the tested guardrail systems. Its strong performance across accuracy, detection capability, and jailbreak resistance positions it as a highly effective option for enterprise environments requiring dependable security controls.

While other solutions offer partial strength in specific areas, their variability across benchmarks underscores the need for a more robust and comprehensive defense. AIShield Guardian’s balanced performance makes it particularly suited for organizations seeking a resilient and production-ready guardrail strategy.

Despite incorporating advanced safety training and a built-in “safe completion” mechanism, GPT-5 still fails to reliably detect certain unsafe or adversarial prompts*. Recent research has uncovered a critical flaw: GPT-5 internally routes some user queries to smaller (weaker) models to save resources [16]. Attackers can exploit this by including trigger phrases that force the router to select an older model. In practice, this means a malicious user could prepend a special instruction that causes GPT-5’s router to hand the request off to a less-safe model (like GPT-3.5 or a “mini” variant). Those weaker models have weaker internal reasoning thus resulting in previous blocked jailbreak prompts to succeed when executed their Security analysts labeled this “PROMISQROUTE” and showed it re-enables old jailbreak attacks.

Moreover, independent reviews of GPT-5 caution that it is not infallible. An in-depth system card analysis notes that although GPT-5 is “more capable and safer than [earlier] models,” it still requires external safeguards: “ship with independent evals, continuous red-teaming, and strict policy controls” [17]. This means even the latest AI has gaps. In our evaluation, GPT-5 used in a “judge” role was still susceptible to overlooking nuanced policy violations and could be manipulated through carefully crafted prompts. These observations illustrate that an LLM’s built-in safeguard alone cannot provide adequate protection for security-critical environments.

*For a comprehensive report on GPT-5 AIShield LLM Red Teaming, please contact to AIShield