Overview
A set of Android and cross-platform apps with integrated LLM features — conversational chatbots, image understanding, voice interaction and smart recommendations — built for real users rather than demos.
The problem
Clients wanted genuine AI capabilities in their apps without the fragility that comes from calling a model directly from UI code. The challenge was reliability: AI calls fail, latency varies, and costs need watching.
Approach
- Wrapped all model access behind an
AiServiceseam that owns prompts, retries, caching and fallbacks. - Implemented streaming responses so users see progress immediately.
- Built graceful non-AI fallbacks so apps stay usable when the model is unavailable.
- Added on-device classification for cheap tasks and cloud models only where needed — cutting cost and keeping things fast.
- Instrumented the AI path with success rate, latency and fallback frequency.
Result
Shipped AI features that behave predictably in production — responsive under load, resilient to outages, and cost-controlled through hybrid on-device / cloud routing.