Local or cloud, the honest arithmetic

The calculation everybody makes goes like this: the graphics card costs this much once, the cloud costs this much per month, after this many months the card has paid for itself. That calculation is almost always wrong, because it leaves out two items that are larger than the hardware price.

The first is your working time. The second is the quality gap, which shows up as longer rework. With both included the result looks different, and for some use cases it then tips the other way. This playbook does the arithmetic honestly, in both directions.

1. Sort by the reason first, not by the price

There are three reasons for local, and only one of them is a maths problem.

If the reason is privacy, the arithmetic is secondary. Data that must not leave the building must not leave it even if the cloud were cheaper. Here the requirement decides, not the money.

If the reason is independence, you are buying insurance. No provider changing prices, no model suddenly switched off, no block because of a policy. That has a value you can put a number on, but not in token prices.

If the reason is cost, then do the maths. The next steps do exactly that.

Tip: If you notice that your actual reason is curiosity, that is entirely legitimate. Just do not do the maths then, set yourself a budget instead. That is more honest and ends better.

2. Measure your real usage before you calculate

Almost everybody overestimates their usage. The impression of "we use AI constantly" comes from how often it is used, not from the volume of tokens.

Look up in your provider's billing what the last three months actually cost. That number is the starting point. Everything else is gut feeling, and gut feeling is unreliable with token prices because the amounts look so small.

Tip: Separate two things that get mixed up here. A flat-rate plan covers interactive use inside that app, with usage limits and fair-use rules, and it is not meant as a channel for automated bulk processing. For anything automated you pay API prices per token alongside it. Throw both into one pot and you either flatter or punish local hardware artificially.

3. Count the full hardware, not just the graphics card

People buying for local models rarely buy only the card. A power supply that can carry it comes along, often a different case, sometimes more RAM, and for larger setups a machine that is allowed to run overnight.

With Apple machines it is easier to see, because the memory is fixed at purchase. On the other hand it cannot be upgraded, which makes the decision more final.

Tip: Calculate with three years of service life, not five. At the pace models and requirements move, a card is not broken after three years, but it is too small for what has become normal by then. There is a residual value, though, graphics cards sell on second-hand, and you can offset that.

So the full calculation looks like this: (hardware minus residual value) divided by the months of service life, plus electricity per month, plus your maintenance time times your hourly rate, plus the rework from step 6. That sum is what you hold against the monthly cloud bill, not the purchase price alone.

4. Include electricity

A capable graphics card draws several hundred watts under full load. At German electricity prices that is not a rounding error, it is a line item.

The calculation is simple: power draw in kilowatts, times hours under load per month, times your price per kilowatt hour. The second number decides, and most people badly overestimate it, because the card only pulls full power under load. Count idle anyway if the machine runs continuously for the purpose: a waiting machine does not draw zero watts, and over a month that adds up.

Tip: If the machine is running anyway and you only ask something occasionally, the electricity item is small. If a machine runs permanently for bulk processing, it is large. Those two cases are economically completely different.

5. Put in your working time, honestly

This is the item missing from most comparisons and the one that most often tips the result.

A local setup wants installing, updating and repairing when something breaks. Models get replaced, runtimes get updates, occasionally something stops working after an update. Count on a few hours at the start and recurring effort after that.

Use your own hourly rate. If you find that setting it up costs more than two years of cloud usage, that is a result and not an argument against doing it anyway. It just has to be a conscious choice.

Tip: Whoever wants to learn the subject anyway may put this item lower, because part of it books as training. Whoever just wants a tool may not.

6. Put a number on the quality gap

A mid-range local model is close to the big providers on simple text tasks. On hard tasks it is not, and that difference costs time, because you rework more.

Take ten typical tasks and have both handle them. Do not measure quality in points, measure the minutes you still need after the answer until the result is usable. That difference times frequency is a real cost item.

Tip: This exact test decides the question better than any leaderboard. The procedure is in Which local model fits your hardware, step 7.

7. Check whether the cloud side is even calculated correctly

Many people compare local hardware against the most expensive cloud model. That is unfair to the cloud.

If your task is simple enough that a local model with eight billion parameters manages it, then the budget model in the cloud manages it too. And that costs a fraction of the top model. The honest comparison is local against budget tier, not local against flagship.

Tip: A setup that routes simple requests to a cheap model and only passes the hard ones upward is the biggest cost lever there is, and it needs no hardware at all.

8. Recognise the cases where local clearly wins

There are patterns where the arithmetic comes out unambiguously.

Very high volume on a simple task. Classifying tens of thousands of documents, sorting texts into categories, normalising masses of records. Here the cloud charges per item and your own machine simply runs through.

Data that must not leave. Already covered, here the requirement decides.

No reliable connection. Anyone who has to work offline has no choice.

Constant load rather than spikes. A machine that is continuously busy amortises. One that waits 95 percent of the time does not.

Tip: If none of these four patterns applies to you and privacy is not an issue, the cloud very probably wins the arithmetic.

9. Recognise the cases where the cloud clearly wins

That belongs in an honest account too.

With fluctuating usage you only pay for what you consume in the cloud, whereas hardware depreciates in quiet months as well. On hard reasoning tasks the quality gap is large enough that the rework costs more than the token price. In small teams without IT the operating effort is the real cost driver. And whoever needs the latest model generation gets it in the cloud on release day and locally months later in a smaller form.

Tip: A flat-rate plan shifts this calculation further, because one more interactive request costs nothing extra as long as you stay inside the usage limits. Above them, throttling or an add-on tariff kicks in, and for automated runs the plan does not apply at all. So anyone on one should build locally mainly for the reasons in step 1, and for cost reasons only where real volume actually accumulates.

10. Choose mixed if you are unsure

The two camps discuss this as though you had to pick a side. You do not, and most productive setups do not either.

A mixed setup sends everything with sensitive data and all bulk work to the local model, and everything hard to the cloud. You get privacy where you need it and quality where it counts.

There is one thing you must not economise on, or you undo exactly the privacy you are building for: two separate environments, not one switch. The sensitive tools, histories and knowledge stores belong solely in the local configuration, the cloud credentials solely in the other. The same interface with a model selector is not enough: whatever a read tool once pulled into the history travels out with you when you switch.

Tip: It being the same server software either way saves you the setup of the second environment, not the separation itself. Which servers you register in each remains a security decision. The detail is in MCP with local models, step 10.

What next

If you have nothing running yet and want to do the arithmetic with your own experience first, Your first local AI model in 30 minutes takes exactly as long as it says. If privacy is your main reason, Local AI and the GDPR sorts out what local really solves. And if you stay in the cloud but want to push the bill down, Cost controls for daily drivers is the more direct lever.

Source

Prices and model line-ups change quickly. The method above stays valid, the numbers you have to plug in fresh each time.

• Anthropic, current pricing per model: https://www.anthropic.com/pricing
• OpenAI, pricing overview: https://openai.com/api/pricing
• Ollama, model library with size figures: https://ollama.com/library