Guardrails AI is an AI reliability platform and open-source framework designed to ensure the reliability of large language model (LLM) applications. It functions as an orchestration and validation engine that wraps LLM calls to intercept and apply checks on both inputs and outputs across different models and deployment environments. The platform provides capabilities for both pre-deployment simulation and runtime risk mitigation. For pre-deployment, it enables the testing of AI models by simulating conversation datasets and generating dynamic test scenarios to identify edge cases. During runtime, it provides low-latency, real-time policy enforcement for LLM applications that handle both unstructured strings and structured data schemas.
The system is accessible via a Python framework and a REST API with an OpenAI SDK-compatible endpoint, operating through a client-server model that decouples validation from the application for independent scaling. It utilizes a hub of modular validators to detect and mitigate risks such as personally identifiable information (PII) leaks, hallucinations, jailbreaks, and custom policy violations. When an output fails validation, the system triggers configurable corrective actions, including re-asking the LLM or programmatically fixing the response. The framework supports asynchronous behavior and streaming for non-blocking validation, and provides execution history logging and OpenTelemetry integrations for monitoring.
