Glossary

AI release testing terms.

• Action confirmation Requiring explicit user approval before side effects.
• Adversarial prompt A hostile or crafted user input meant to break intended AI behavior.
• AI agent A workflow where an AI decides what to do next, often across multiple steps.
• AI API An endpoint your product calls to get model output.
• AI release process How your team decides an AI feature is ready to ship.
• AI workflow A multi-step automated process involving AI decisions.
• Attack pack A focused bundle of adversarial tests for one risk area.
• Attack surface Where untrusted input can influence AI behavior.
• Baseline scan The first scan you treat as a reference point for later builds.
• Blocked Release Gate state: critical issues must be resolved first.
• Boundary bypass When users push an AI past intended scope or policy limits.
• Chatbot A conversational AI surface users interact with in natural language.
• Critical finding A finding that usually blocks release under default policy.
• Customer-safe evidence Finding evidence redacted for sharing outside core team.
• Data exfiltration When an AI reveals private data, secrets, or internal context it should not share.
• Evidence Customer-safe details showing what failed in a scan.
• False negative When a test passes or finds nothing, but a real break still exists.
• Finding A grouped scan result describing a failure pattern with severity and next steps.
• Fix Release Gate state: address findings before release.
• Grounded answer A reply tied to retrieved or cited source material.
• Hallucination When an AI states something confidently without reliable support.
• Hidden instructions Rules or setup text meant to stay private but exposed under pressure.
• High finding A serious issue that often requires fix or strong monitor plan.
• Jailbreak When a user gets an AI to ignore safety rules, refusals, or scope limits.
• Knowledge retrieval Fetching documents or data to inform an AI reply.
• Low finding A minor issue or edge case to track.
• MCP agent An AI workflow that calls tools through the Model Context Protocol.
• Medium finding A meaningful issue that may be acceptable with monitor plan.
• Monitor Release Gate state: ship with known risks to watch.
• Permission abuse Attempts to perform actions beyond the user role allowed.
• Pressure test Adversarial testing that hits your AI app the way a motivated user or attacker would.
• Probe One adversarial test input sent to your target during a scan.
• Prompt injection When untrusted input tries to override intended AI behavior.
• RAG Retrieval-augmented generation: answers grounded in retrieved documents or knowledge.
• Ready Release Gate state: no blockers under current policy.
• Red Teaming Proactive adversarial testing to find breaks before real users do.
• Release blocker A finding or policy rule that stops a Ready recommendation.
• Release importance How costly a bad release would be for this feature.
• Release policy Team rules that shape how findings affect Release Gate recommendations.
• Release report A shareable snapshot of gate state, findings summary, and decision context.
• Report redaction Removing or masking sensitive evidence in shared reports.
• Reproduction Steps to see the same failure again.
• Retest A focused re-run to verify whether a finding is fixed.
• Retrieval drift When answers diverge from or overreach what sources support.
• Retrieval poisoning When hostile or misleading content in a knowledge base steers RAG answers wrong.
• Risk Area A category of failure like injection or unsafe actions.
• Scan An adversarial test run against a target using selected attack packs.
• Scan comparison Side-by-side review of two scans on the same target.
• Scan coverage Which attack packs and scenarios ran in a scan snapshot.
• Scan history Past scans on a target for comparison and trend review.
• Sensitive context exposure When private guidance or context appears in user-visible output.
• Severity How urgent a finding is: critical, high, medium, or low.
• Shared release report A report accessed via share link with optional redaction.
• Ship Decision A ship recommendation based on scan findings and policy: Ready, Monitor, Fix, or Blocked.
• System instruction extraction Pressure to reveal setup instructions or hidden constraints.
• Target The AI app or endpoint Agnostics connects to for scans.
• Tool misuse When an agent calls the wrong tool, skips checks, or uses a tool with unsafe arguments.
• Tool-using agent An AI workflow that can invoke tools or take external actions.
• Unsafe tool action An action attempt that should require confirmation or stronger auth.
• User input boundary The line between trusted system rules and untrusted user content.
• Validation A pre-scan check that Agnostics can reach your target.