Glossary
AI release testing terms.
- Action confirmation Requiring explicit user approval before side effects.
- Adversarial prompt A hostile or crafted user input meant to break intended AI behavior.
- AI agent A workflow where an AI decides what to do next, often across multiple steps.
- AI API An endpoint your product calls to get model output.
- AI release process How your team decides an AI feature is ready to ship.
- AI workflow A multi-step automated process involving AI decisions.
- Attack pack A focused bundle of adversarial tests for one risk area.
- Attack surface Where untrusted input can influence AI behavior.
- Baseline scan The first scan you treat as a reference point for later builds.
- Blocked Release Gate state: critical issues must be resolved first.
- Boundary bypass When users push an AI past intended scope or policy limits.
- Chatbot A conversational AI surface users interact with in natural language.
- Critical finding A finding that usually blocks release under default policy.
- Customer-safe evidence Finding evidence redacted for sharing outside core team.
- Data exfiltration When an AI reveals private data, secrets, or internal context it should not share.
- Evidence Customer-safe details showing what failed in a scan.
- False negative When a test passes or finds nothing, but a real break still exists.
- Finding A grouped scan result describing a failure pattern with severity and next steps.
- Fix Release Gate state: address findings before release.
- Grounded answer A reply tied to retrieved or cited source material.
- Hallucination When an AI states something confidently without reliable support.
- Hidden instructions Rules or setup text meant to stay private but exposed under pressure.
- High finding A serious issue that often requires fix or strong monitor plan.
- Jailbreak When a user gets an AI to ignore safety rules, refusals, or scope limits.
- Knowledge retrieval Fetching documents or data to inform an AI reply.
- Low finding A minor issue or edge case to track.
- MCP agent An AI workflow that calls tools through the Model Context Protocol.
- Medium finding A meaningful issue that may be acceptable with monitor plan.
- Monitor Release Gate state: ship with known risks to watch.
- Permission abuse Attempts to perform actions beyond the user role allowed.
- Pressure test Adversarial testing that hits your AI app the way a motivated user or attacker would.
- Probe One adversarial test input sent to your target during a scan.
- Prompt injection When untrusted input tries to override intended AI behavior.
- RAG Retrieval-augmented generation: answers grounded in retrieved documents or knowledge.
- Ready Release Gate state: no blockers under current policy.
- Red Teaming Proactive adversarial testing to find breaks before real users do.
- Release blocker A finding or policy rule that stops a Ready recommendation.
- Release importance How costly a bad release would be for this feature.
- Release policy Team rules that shape how findings affect Release Gate recommendations.
- Release report A shareable snapshot of gate state, findings summary, and decision context.
- Report redaction Removing or masking sensitive evidence in shared reports.
- Reproduction Steps to see the same failure again.
- Retest A focused re-run to verify whether a finding is fixed.
- Retrieval drift When answers diverge from or overreach what sources support.
- Retrieval poisoning When hostile or misleading content in a knowledge base steers RAG answers wrong.
- Risk Area A category of failure like injection or unsafe actions.
- Scan An adversarial test run against a target using selected attack packs.
- Scan comparison Side-by-side review of two scans on the same target.
- Scan coverage Which attack packs and scenarios ran in a scan snapshot.
- Scan history Past scans on a target for comparison and trend review.
- Sensitive context exposure When private guidance or context appears in user-visible output.
- Severity How urgent a finding is: critical, high, medium, or low.
- Shared release report A report accessed via share link with optional redaction.
- Ship Decision A ship recommendation based on scan findings and policy: Ready, Monitor, Fix, or Blocked.
- System instruction extraction Pressure to reveal setup instructions or hidden constraints.
- Target The AI app or endpoint Agnostics connects to for scans.
- Tool misuse When an agent calls the wrong tool, skips checks, or uses a tool with unsafe arguments.
- Tool-using agent An AI workflow that can invoke tools or take external actions.
- Unsafe tool action An action attempt that should require confirmation or stronger auth.
- User input boundary The line between trusted system rules and untrusted user content.
- Validation A pre-scan check that Agnostics can reach your target.