Gloo AI Gateway by Solo.io is a cloud-native gateway built on Envoy for securing, observing, and controlling AI applications. It accelerates AI innovation by eliminating development friction and boilerplate code, and provides features like prompt templating, multi-LLM provider support, and prompt retries. It also protects applications, models, and data from inappropriate access. Gloo AI Gateway ensures safe use of AI with governance controls, auditability, and visibility into consumption. It leverages advanced AI integration patterns for data augmentation and integration with cloud-native gateway capabilities to support high-volume, zero-downtime AI connectivity. Gloo AI Gateway includes features such as prompt guard and auditing, data exfiltration protection, and token-weighted rate limiting. It supports Retrieval Augmented Generation (RAG), model fallback/failover, and semantic caching. Gloo AI Gateway is designed to manage credentials and access control to secure AI workloads, implement prompt management and guards to improve response accuracy and protect sensitive content, and control token usage and build resiliency through rate limiting and model failovers. It also optimizes API performance with semantic caching and RAG, and provides a flexible, multi-tiered architecture to run as a central enterprise gateway and as a distributed micro-gateway. Gloo AI Gateway pairs seamlessly with Istio for complete connectivity and observability, and integrates with leading technologies like Backstage and Kubernetes Gateway API to provide modern platform engineering and cloud teams with the tools and solutions they want to use.
