AI Agents Can Increasingly Run Locally Instead of Sending Everything to the Cloud

Many students first encounter AI application development through a simple pattern: send a request to a cloud model and use the response. Local model support is making that architecture more flexible.

Google Developers announced local-model support in the Antigravity SDK on 23 September. Initial support includes the Gemma 4 26B A4B checkpoint through Google AI Edge’s LiteRT runtime, alongside support for local OpenAI-compatible servers such as Ollama, LM Studio and vLLM.

Google highlights four reasons to consider local execution: keeping sensitive information on a device, avoiding API costs and rate limits, operating without a network connection, and combining local and cloud models in one workflow.

Local does not automatically mean simple

A hybrid application might use a cloud model for high-level planning while a local model processes source code or performs repetitive tasks. That can reduce how much sensitive material leaves the machine while reserving larger cloud models for work that benefits from them.

There are practical limits. Google recommends at least 24 GB of VRAM or unified memory for the highlighted Gemma checkpoint, whose download is approximately 16.8 GB. Hardware capacity, model quality, latency, maintenance and energy use all influence whether local execution is suitable.

This turns model selection into an architecture decision. An AI engineer may need to ask:

• Does this task contain private data?
• Can the available hardware run the model reliably?
• Does the task need the capability of a cloud model?
• What happens when the network is unavailable?
• Which routing choice gives an acceptable balance of quality, cost and speed?

The future is unlikely to be entirely local or entirely cloud-based. Different tasks can move between models and environments according to their requirements.

Takeaway

AI engineering is expanding beyond choosing a model or writing an API call. Engineers increasingly need to decide where inference should happen and design routing, privacy, cost and fallback behaviour around that choice.