Local AI on your own machine, so your data never leaves the house

There's a point where almost everyone who starts working seriously with AI gets stuck. You want to have a contract summarized, a payslip, a sick note, customer data. And then you pause for a second, because that stuff goes to a server in the US the moment you hit Enter. For me, that was the moment I started taking local models seriously.

Local AI means: the model runs on your own machine. No account, no cloud, no data leaking out. What you type in stays on the hard drive. Two years ago that was still tinkering with a command line and CUDA error messages. Today it's one click. Ollama cracked 52 million downloads a month in 2026 and now ships a proper desktop app with a chat window, LM Studio has become completely free. You don't need to be a developer.

Here's the order I give people doing this for the first time. Ten steps, once through is a good half hour, after which you have an AI that runs offline.

Step 1, understand what local actually changes

The difference isn't the answer quality, that's still clearly better with the big cloud models like Opus 4.8 or GPT-5. The difference is where your input ends up. With ChatGPT or Claude, every word travels over your connection to someone else's computer, gets processed there and usually stored for a while too. With a local model, the whole computation happens in your living room. You can unplug the Wi-Fi and it keeps working.

That makes local AI the honest answer to the question that nags at you around data protection. Instead of memorizing what you must never type into a cloud AI, you simply put sensitive things into the local model. What never leaves the house can't leak out either.

Step 2, check what your machine can handle

Local models need memory, and noticeably so. The rule of thumb is simple. If you have 8 GB of RAM and no dedicated graphics card, meaning a typical office laptop or a MacBook Air, then small models with 3 to 4 billion parameters run well. If you have 16 GB of RAM or a graphics card with 8 GB, 7 to 8 billion parameters run smoothly. From 24 GB on, you can run the really big stuff.

On Apple machines with an M chip, the shared memory counts, a MacBook with 18 GB behaves roughly like a graphics card with 12 GB. You don't have to memorize this. The only important thing is to be honest about what you have, so you don't pull a 40 GB model onto an 8 GB laptop and wonder why nothing works.

Step 3, install Ollama, without any terminal at all

Go to ollama.com and download the app for your system. There's a native version for macOS and for Windows, both with a graphical interface. On Windows you don't even need administrator rights, the app installs into your user directory. Double-click, click through, done.

After installation Ollama runs in the background. From now on a small service on your machine is listening at localhost:11434, that's the address everything runs through. You don't have to configure anything there, it's just noted here so you recognize it if you see it somewhere.

Step 4, load the first model

To start with, take gemma3. That's Google's open model, it can do 140 languages and is therefore noticeably better at German than many other small models. If you have a weaker laptop, take the small variant with 4 billion parameters, if you have more memory the 12 billion one.

In the desktop app you simply type the model name into the chat field or pick it from the list, then Ollama downloads the file. Reckon with a few gigabytes, the first time that takes a minute or two depending on your connection. After that the model sits on your drive and is never loaded again.

Step 5, chat like in ChatGPT

Now comes the part that used to be missing. The Ollama app has a normal chat window. You type your question in, hit Enter, the model answers. You can drag and drop files and images in if you're using a model that can see. It feels like ChatGPT, except that there's no line going out to the internet underneath.

Try it with something concrete. Throw in a real sick note and have it summarized, or an invoice, or a long customer text. Exactly the stuff you'd never want to hand to a cloud. And notice how it feels that the answer comes more slowly than with cloud AI. That's normal, your machine isn't a data center.

Step 6, the terminal route, if you like it

Some people prefer the command line, and for them it's just as easy. A single command loads a model and starts the chat: ollama run gemma3. If you only want to download without chatting, use ollama pull qwen3:4b. The colon specifies the size, qwen3 in the 4 billion variant is a good second candidate alongside gemma3.

The nice thing about it, every model you load this way also shows up in the desktop app, and vice versa. Terminal and app draw on the same collection. So you don't have to commit to one route.

Step 7, choose the model to fit the task

There isn't one single best local model, there's the right one for the task and your hardware. For general writing and summarizing in German, gemma3 is the safe choice. For translations and multilingual work, qwen3 is strong. If you code, there are dedicated coder models like qwen2.5-coder.

Small models are fast but more forgetful and less precise, big ones are smart but slow and memory-hungry. A good starting point is a 4 billion model, that runs almost anywhere. If the answers feel too thin to you and your machine can handle it, step up to 7 or 8 billion. You can have several models installed in parallel and switch depending on the task.

Step 8, be honest about where local hits its limits

So you don't end up disappointed, here's the truth. A local model on your laptop isn't as smart as Opus 4.8. It can't look things up on the web. It has no current news, its knowledge ends where its training stopped. And on long, complicated tasks it makes more mistakes than the big cloud models.

That's not a reason to skip it, that's a reason to use it right. For sensitive routine work, summarizing, rephrasing, first drafts, translating, a local 8 billion model is more than enough. For the hard thinking work you keep using the cloud.

Step 9, combine local and cloud cleverly

The best setup is usually not an either-or. You run two tools side by side. Everything personal, confidential or that smells of data protection goes into the local model. Everything where you need the best quality and there's no sensitive data in it goes into the cloud.

In practice that means you do the first pass with a real contract locally, strip out the names and figures, and then send only the anonymized rest to the big AI when you need a particularly good answer. That way you get quality without the sensitive material ever leaving the house.

Step 10, check the data protection yourself, just once

Don't just trust that local is really local, test it. The test takes a minute. Start a chat with your local model, and cut the internet connection while you do it, Wi-Fi off, cable out. If the answer still comes, you know for sure, it ran on your machine. Nothing went out.

That's the point that sets local AI apart from any data protection promise a cloud company gives you. You don't have to believe anyone. You can check it yourself, by pulling the plug and it keeps working.

What's next

If you want to know why the cloud models are smarter anyway and exactly where that difference comes from, the lesson on the model landscape in Level 1 is worth it. And if you want to build local AI into your working day, without rethinking at every sensitive task what you're allowed to enter, the next step is a fixed habit, sensitive stays local, uncritical is allowed into the cloud. That dividing line is exactly the core of clean AI data protection.

Source

Model names, commands and the address localhost:11434 verified against the official Ollama documentation (docs.ollama.com/windows), as of July 2026. The details on the desktop app and the model catalog come from the same source as well as the Ollama model overview 2026.