Can AI run on my own computer?

At some point the question comes up. Usually the first time you hesitate before typing something into a chat window. A medical letter, a reference from an employer, the draft of a resignation, numbers from your own company. You type it, pause for a second and think: do I really want to upload this?

The good news is that there is an alternative. Language models also run on ordinary computers. Not the very largest ones, but surprisingly usable ones. This lesson explains what that is, what it can and cannot do, so you can decide for yourself whether it makes sense for you.

What a local model actually is

When you use ChatGPT, you type into your browser and the actual computation happens in a data centre. Your text travels there, the model answers, the answer travels back. That is fast and feels as though it were happening on your machine.

A local model is a file on your hard drive. A few gigabytes in size, containing the weights, meaning what the model learned during training. When you ask a question, your own computer does the maths. You can unplug the network cable and it keeps working.

One qualification straight away, because it matters later: what is local is the model, not automatically the whole program around it. Many interfaces can additionally search the web or have documents indexed by a cloud service. With that switched on, parts of your input do go out. It can be turned off, you just have to know it is there.

That is the whole difference, and it is simultaneously bigger and smaller than people think. Bigger on privacy, because with a properly configured program there is no transmission to a provider at all. Smaller in operation, because a local chat feels exactly like any other.

Why this is no longer a niche

Two years ago this was for tinkerers. You had to compile things, convert models into odd formats and bring a lot of patience.

Today you download a program, click a model and chat. The reason for that leap is not better hardware, it is that the models have become considerably more efficient. A model that fits into eight gigabytes today is better than one that needed four times that eighteen months ago.

On top of that, large providers now offer their models for download themselves. Google, Alibaba, Meta, Mistral and OpenAI too publish models with open weights. That is the reason there is any choice at all today. What you may do with them is governed by the respective licence, and that differs from model to model. It applies privately too, it just surfaces less often there.

What your machine needs for it

The decisive quantity is memory, not processor speed.

On a Windows machine with its own graphics card, the memory on that card counts. On a Mac from the last few years the number on the box counts, but only part of it goes to the model, because processor and graphics share the same memory. Without a graphics card it works too, the main processor then does the maths and it takes longer.

Roughly placed, as of August 2026, and meaning usable memory in each case: from eight gigabytes a small model runs. From sixteen it gets comfortable. From twenty-four it gets serious on substance. On a Mac that means buying one size up from what the number suggests. If you bought a decent machine in the last three or four years, you can probably try it.

What it is genuinely good for

Here it gets concrete, and here it is worth being honest, because expectations that are too high are the most common reason people give up after two days.

What works well is anything that operates on a text you already have. Summarizing, rewriting, translating, changing the tone, pulling the key points out of a long document, sorting texts into categories, turning unstructured notes into a list. That is the diligence half of text work, and a local model handles it decently.

Also good: questions about a document you attach. Exactly the case where you hesitated earlier.

A word of caution, because two things get confused here. That the data never leaves your machine does not make the answer any more correct. With a medical letter, a contract or an official decision, a summary is a comprehension aid and not a reliable reproduction: negations, dosages, deadlines and amounts in particular do get garbled by small models. Take the summary as a way in, read the decisive passages in the original, and do not base a decision on it alone.

Where it clearly loses

It gets difficult wherever real thinking is required. Long chains of reasoning where step seven builds on step three. Demanding programming work where the structure of a whole system has to stay in view. Working independently across many steps.

There the big models are not slightly better, they are clearly better. And that will not change next week either, because it is the price of something running on a device under your desk instead of in a hall full of specialised hardware.

General world knowledge is also thinner. A small model knows less and invents things somewhat more often to compensate. So the limits from the lesson What AI cannot do do not merely apply here, they are more pronounced.

The point where it gets interesting

There is one thing that partly offsets the size difference, and it is the thread running through this academy.

A model that knows your context beats a stronger one that has to guess. If you give a small local model the right documents, it is better on questions about your project than a large model that does not know your project. Not because it is smarter, but because the answer is in the documents and does not have to sit inside the model.

That is exactly why local and memory are such a good combination. From level 4 onwards that is the subject in depth.

When the effort is worth it

Four cases where it clearly pays off.

If you regularly work with data that must not go outside. That is the strongest reason and for many professions the only one that counts.

If you process a great deal of similar material. Sorting thousands of texts costs per item in the cloud, on your own machine it simply runs through.

If you often work without a reliable connection.

And if you want to understand how all of this works. A model whose file you can see and delete demystifies the thing in a useful way.

If none of these four applies, it is entirely fine to stay with the cloud. Most productive setups are mixed anyway: the local one for the sensitive and the bulk, the big models for the hard cases.

How to try it without breaking anything

It is reversible. You install a program, download a model, try it for a week, and if it is not for you, you remove it again. Bear in mind that the model and your uploaded documents live in different places and both have to go; the playbook shows the full path. No account, no subscription, no notice period, just a few gigabytes of disk space that you get back.

The step-by-step guide is in the playbook Your first local AI model in 30 minutes. If you want to know beforehand which model suits your machine, that is covered by Which local model fits your hardware.

Quiz

1. What is the main advantage of a local model over the cloud?

• A) It is smarter than the big models
• B) Your data does not leave the machine
• C) It always answers faster

B is correct, provided the program around it is configured fully locally. It is not smarter, and usually not faster either. The advantage is that there is no transmission to a provider, as long as web search and cloud services in the interface are switched off.

2. Which number mainly decides which model runs on your machine?

• A) The processor clock speed
• B) The size of the hard drive
• C) The available system or graphics memory

C is correct. The model has to sit in memory while it answers. The hard drive only limits how many models you have lying around at once.

3. Which task suits a small local model best?

• A) Designing the architecture of a complex piece of software
• B) Translating a confidential medical letter into plain language
• C) Running a multi-stage research task independently

B is correct. Working on a text you already have is the strength, and the confidential content is exactly why local wins here. What stays true: the translation is a comprehension aid, and you read the decisive passages in the original. A and C need deep thinking across many steps, and there the small model clearly loses.

Source

Figures on memory requirements are approximations, as of August 2026, and change with every model generation.

• Ollama, getting started and model library: ollama.com
• LM Studio, interface for local models: lmstudio.ai
• Hugging Face, model cards with licence and size: huggingface.co/models