Stoplight
Design, document, and build APIs faster.

Empowering NPCs with human-like conversation and intelligent spatial awareness in virtual worlds.

Convai is a specialized AI orchestration platform designed to provide real-time conversational intelligence for non-player characters (NPCs) and virtual assistants within 3D environments. By 2026, its architecture has evolved into a highly optimized pipeline that bridges Large Language Models (LLMs) with gaming engines like Unity, Unreal Engine 5, and NVIDIA Omniverse. The platform's technical core is built around a low-latency Speech-to-Speech (S2S) engine that handles STT (Speech-to-Text), LLM processing, and TTS (Text-to-Speech) in a unified stream, reducing perception-response time to sub-200ms levels. Beyond simple dialogue, Convai distinguishes itself through its Action API and Spatial Awareness features, which allow characters to perceive their environment and execute programmatic actions based on natural language commands. This enables a level of agency where an NPC can navigate a map, identify objects, and interact with the game world's physics or logic systems autonomously. Positioned as a mission-critical middleware for the Metaverse and high-fidelity training simulations, Convai provides the infrastructure for persistent, memory-capable digital entities that evolve through user interaction.
Convai is a specialized AI orchestration platform designed to provide real-time conversational intelligence for non-player characters (NPCs) and virtual assistants within 3D environments.
Explore all tools that specialize in rag querying. This domain focus ensures Convai delivers optimized results for this specific requirement.
A semantic mapping engine that converts user verbal requests into structured JSON triggers for game engine logic.
Enables NPCs to query the 3D bounding boxes and metadata of nearby objects in the game scene.
Vector database integration that stores past interactions, allowing characters to recall previous conversations across sessions.
Generates viseme and blend-shape data in real-time synchronized with the audio stream.
An interface that allows the AI to receive a text-based summary of the 3D environment layout.
Integration with ElevenLabs and proprietary models for custom character voices.
A unified character state managed on the cloud, accessible via REST or gRPC across different engines.
Sign up for a Convai developer account and obtain your API Key.
Create a new Character in the Convai Dashboard.
Define the character's 'Backstory' and 'Core Personality' to guide the LLM's response style.
Upload knowledge base documents (PDF/TXT) to the character for RAG-based domain expertise.
Define 'Actions' in the dashboard that the NPC can trigger (e.g., 'Open Door', 'Follow Player').
Download the Convai SDK for your specific engine (Unity, Unreal Engine, or Web).
Import the SDK and enter your Character ID and API Key in the engine's global settings.
Map the SDK's lip-sync and skeletal animation outputs to your 3D character model.
Configure the spatial awareness sensor on the NPC to detect nearby game objects.
Test interactions in-editor using the push-to-talk or text-input components.
All Set
Ready to go
Verified feedback from other users.
"Users praise the extremely low latency and the power of the Action API, though some note a steep learning curve for complex spatial integration."
Post questions, share tips, and help other users.
Design, document, and build APIs faster.
Digital developers who are actually easy to work with.
Open Source LLM Engineering Platform

The Open-Source Framework for Reinforcement Learning in Quantitative Finance.

Enterprise-grade Python library for modular backtesting and quantitative financial market analysis.

Static bytecode analysis to identify potential defects and vulnerabilities in Java applications.