Google Quietly Open-Sourced a Monster: You Can Now Run a Free AI Assistant on Your Old PC

Google quietly released Gemma 4 on April 2, 2026, and it barely caused a ripple in the AI news cycle. Buried under the avalanche of flashier model launches, this release was easy to miss. Yet for the aveRAGe user, Gemma 4 is genuinely more prACTical than most of the headlining models. The reason is simple: you can download it, run it entirely on your own hardware, and never pay a cent. Your data stays local, and the license—Apache 2.0—actually lets you use it freely.

Someone already got the E4B variant running on a consumer-grade RTX 3060 with 6GB of VRAM. It works at a decent speed without crashing. That alone is a signal that local AI has crossed an important threshold.

What It Actually Is, Without the Hype

Let’s cut through the marketing fluff. A 4B-parameter activated model is not “rivalling flagship models” in raw Intelligence, and anyone claiming otherwise is misleading you. Gemma 4 comes in four sizes: E2B (2.3B parameters), E4B (4.5B parameters), 26B-A4B (a Mixture-of-Experts architecture with 4B activated parameters), and a 31B Dense model. For most people, the choice is straightforward: pick E4B.

E4B demands only 6–8GB of VRAM or an Apple M-series chip with sufficient unified mEMOry. A real-world test posted on CSDN on April 9 confirmed stable deployment on a Windows machine with an RTX 3060. Response speed on common tasks is perfectly usable—you’re not waiting long enough to go pour a coffee and come back. For content drafting, document organization, Copywriting, or translation, E4B is entirely adequate.

One honest caveat: it’s not the strongest model for Chinese-language tasks. If polished Chinese writing is your primary need, use Qwen3 instead. Where Gemma 4 excels is multimodality and tool use: it processes images, underStands audio, and supports a context window ranging from 128K to 256K tokens. Its real strengths lie in analyzing long documents, interpreting charts, and plugging into automated workflows. It’s not the most linguistically Fluent Chinese model, but it’s arguably the most functionally complete one you can run locally.

How to Install—Seriously, About Five Minutes

Anyone who has deployed local models before knows the drill: wrestling with CUDA version mismatches, Python environment conflicts, and outdated drivers for an entire afternoon. What should feel like adopting a dog turns into building a kennel from scratch, only to discover the lumber is the wrong size.

Gemma 4, specifically through a tool called OLlama, flattens this process entirely.

Step one: install Ollama. Download the installer from ollama.com for Windows, use Homebrew for Mac, or run a single command for Linux:
curl -fsSL https://ollama.com/install.sh | sh
Once installed, it runs in the background requiring zero configuration.

Step two: pull the model. Open a terminal and type:
ollama pull gemma4:e4b
It downloads roughly 2–3GB depending on your network speed. Go make tea.

Step three: run it.
ollama run gemma4:e4b
That’s it. The first time I tried it, I kept waiting for something to break. Instead, a cursor blinked, ready for my message. It felt surreal.

What If You Don’t Have a Dedicated GPU?

You can, but temper your expectations. APPle Silicon Macs (M1/M2/M3) leverage unified memory architecture exceptionally well; a 16GB M-series machine runs E4B at a perfectly usable speed, and many people use exactly this setup daily. On a Windows machine with CPU-only inference, E2B barely manages acceptable performance, and E4B is painfully slow—we’re talking one or two tokens per second. That’s the kind of speed where you watch characters trickle out one by one, like waiting for a kettle to boil, and your patience evaporates first.

If you’re on a gaming laptop with less than 4GB of VRAM, try E2B first to gauge the experience. If the speed frustrates you, switch to a cloud tool. Don’t torment yourself chasing “fully local” on an NVIDIA 1060 just for the sake of it.

One known trap: Ollama auto-detects CPU versus GPU but occasionally gets it wrong, and the issue hasn’t been completely fixed. If you have a dedicated GPU yet performance feels sluggish, set the environment variable CUDA_VISIBLE_DEVICES=0 to force GPU usage. The CSDN test post from April 9 mentioned this, and the comment section was full of grateful users.

What You Can Actually Do with It

Deploying a model is only the beginning. The real value of local AI lies in tasks that are difficult or uncomfortable to perform with cloud tools.

First, handling privacy-sensitive documents. Internal reports, contract drafts, and customer data are things you may hesitate to upload to OpenAI or claude servers, regardless of their data usage assurances. Gemma 4 processes everything on your machine. This advantage isn’t a nice-to-have; for many, it’s a hard requirement.

Second, building a personal workflow engine. Ollama exposes a local API endpoint at localhost:11434 that any compatible tool can connect to—n8n, Dify, and the increasingly popular Flowise all work. You can treat Gemma 4 as your private backend brain, automating tasks like summarizing your emails every morning, organizing meeting notes, or extracting key Information from a stack of reports. Set it up once, and it largely runs itself.

Third, creating a local knowledge base. With a 256K context window, Gemma 4 can digest an entire set of internal company documents and answer direct questions: “Was Supplier X mentioned last quarter?” or “What does the contract say about breach of contract terms?” It’s Dramatically faster than manual searching.

The Honest Gap Compared to Paid Tools

Gemma 4 E4B still trails Claude Sonnet in complex reasoning, nuanced linguistic intuition, and long-form creative writing. There’s no point sugar-coating this—the parameter count is an order of magnitude smaller, and anyone claiming parity has either not tested it properly or is deliberately misleading you.

But “worse in some ways” doesn’t mean “useless.” A crude but accurate analogy: Gemma 4 E4B is your microwave; Claude Sonnet is a Michelin-stARRed restaurant. You don’t go to fine dining to reheat leftovers, and the microwave is yours, available 24/7 without reservation or cost.

The unhealthy fixation some people have—insisting a local model must excel at everything before it’s worth installing—misses the point. Tools divide labor by context. You don’t throw away a screwdriver because it can’t drive nails. The goal isn’t to replace Claude with a local model; it’s to identify which tasks are microwave-safe and reserve the restaurant visits for what truly deserves them. Based on personal experience: document organization, information retrieval, meeting minutes, code commenting, straightforward translation, and format conversion all land firmly in the “good enough” zone. Tasks requiring heavy creative writing, complex logical reasoning, or highly polished Chinese prose are better served by larger models. Shift the adequate tasks to local inference, and reinvest the saved subscription cost into the tasks that genuinely need more powerful tools.

Your Action Plan

If you want to try this, follow this sequence:

1. Download and install Ollama from ollama.com (5 minutes).

2. Open a terminal and run ollama pull gemma4:e4b (wait for the download).

3. Execute ollama run gemma4:e4b and send a test message.

4. If it works, search for “Ollama + Open WebUI” to add a polished chat interface.

5. If speed is too slow, try the E2B version first: ollama pull gemma4:e2b.
   If the GPU isn’t recognized, check your graphics driver version first. Outdated drivers are the single most common culprit, and updating them usually resolves the problem.

A friend confessed last month that he was spending 600 yuan per month on various AI subscription tools. I suggested he test which tasks could run locally and only pay for what couldn’t. He later reported that 30% of his tasks were handled perfectly offline—saving roughly 180 yuan a month. It’s not a fortune, but spending it on something else feels strictly better.