Transcribing conversations offline, without them leaving the building

There is a kind of recording you do not upload. The first meeting with a client, the medical history, the conversation with an employee about a dismissal, the interview with a source who has to stay anonymous. With exactly these recordings a transcript would be worth the most, and with exactly these the usual meeting tools are out.

The good part of this story: speech recognition is the area where going local costs you the least. The models are small, the task is narrow, and the gap to the cloud is smaller than in almost anything else. A machine too weak for text models still transcribes decently. By the end of this playbook you will have a workflow in which the audio file does not leave your machine, and you will know where that workflow reaches its limit.

One sentence up front, because the whole promise rests on it: the model computing locally is not enough for that. Two more things belong to it, the folder and the program, and both are in step 10. Skip them and you have a local transcription sitting in a folder the cloud synchronises.

1. First work out whether your case really has to be local

The effort is not worth it for every recording. If you are having an internal team meeting about the colour of the new logo minuted, take the tool that is already inside your video software. The way there is in AI notes for meetings, and for the normal case it is faster and more convenient.

Local gets interesting as soon as one of these three conditions applies. Special categories of personal data are involved, for example health, ethnic origin, religious belief, trade union membership, sex life, biometric or genetic data. Or professional confidentiality applies, for doctors, lawyers, therapists, tax advisers. Or the other side consented to the recording but expressly not to passing it on to a service provider.

Tip: If you are unsure, test it with one question. Could you tell the person in the conversation to their face that this recording is about to go to somebody else's server? If you hesitate, the case belongs here.

2. Understand what Whisper is and what the licence allows you

The de facto standard for local speech recognition is called Whisper and comes from OpenAI. That sounds like a contradiction at first but is not one: Whisper is published as an open model, code and weights are under the MIT licence. You download it once and from then on run it entirely on your own machine. It does not register anywhere and does not ask for anything.

MIT is the most uncomplicated of the common licences. You may work with it commercially, build it into a product and use it in front of your clients, without anybody dictating conditions about user numbers. There is one condition, though, and it is often glossed over: if you pass on code or weights, the copyright notice and the licence text have to stay with them. That applies to distribution, not to mere use, and it does not require a visible credit in your interface. But anyone who packages Whisper into a product and ships it includes the licence file with it.

Tip: Do not confuse this with the OpenAI speech API that carries the same name. That one runs on somebody else's servers and is exactly what you want to avoid here. What is meant is always the model you download yourself.

3. Choose the model size to match your hardware

Whisper comes in six sizes. The figures come from the official model overview, as of August 2026, and the memory requirement refers to graphics memory.

The smallest tier is called tiny with 39 million parameters and around one gigabyte of demand. base sits at 74 million and likewise about one gigabyte. small has 244 million and needs roughly two gigabytes. medium comes to 769 million and around five gigabytes. large is the heaviest tier at 1.55 billion parameters and about ten gigabytes. In between sits turbo with roughly 800 million parameters and around six gigabytes.

For German-language material start at small and go up if the error rate bothers you. The four smaller tiers additionally exist as an English-only variant with the suffix .en, which is more accurate on English audio and does nothing for you on German.

Tip: turbo is a speed-optimised version of large-v3 and in most cases the best compromise. One detail that is easily overlooked: turbo is not trained for translation. If you want foreign-language audio translated into English rather than transcribed in the original language, you need one of the multilingual tiers.

4. Decide on a runtime

The same model can be executed by different programs, and the difference is considerable. Three routes are common, all three under MIT.

The original version from OpenAI is a Python package. It is the reference point, but the slowest variant. whisper.cpp is a reimplementation in C and C++ without dependencies, which also handles pure processor computation well and uses the graphics unit via Metal on Apple Silicon. faster-whisper builds on the inference library CTranslate2 and is, according to the project, up to four times faster than the original at a lower memory footprint.

How big the gap is shows in a measurement from the faster-whisper project. The same thirteen-minute recording, the same small model, a Core i7-12700K processor with eight threads: the original version needs six minutes and 58 seconds, whisper.cpp two minutes and five seconds, faster-whisper with 8-bit compression one minute and 42 seconds.

Read those numbers carefully, because they do not only compare programs. The original version computes there at full precision, faster-whisper at 8-bit compression. That is part of the speed gain, and the library behind it describes the accuracy loss from this compression as little to none, not as ruled out. Two runs are comparable only once the model, the beam size and the decoding settings are the same.

Tip: If you do not program, take one of the graphical interfaces with whisper.cpp inside instead of compiling it yourself. You are not after a terminal experience, you are after a transcript.

5. Do the first run with real material

The most common mistake at this point is a test with a cleanly spoken sample recording. That always works. Take a real recording from your everyday work instead, with dialect, technical terms, throat-clearing and the colleague who talks over everybody.

Set the language to German explicitly instead of letting it be guessed. Whisper detects the language automatically, and on the first seconds of a recording full of greeting mumble it sometimes gets it wrong. A wrongly detected English then runs through the whole file. In the original version a run looks like this:

whisper recording.wav --model small --language German

The other runtimes from step 4 are invoked differently, but the thinking behind it stays the same. What else your version can do is shown by whisper --help.

Tip: Whisper is often stronger on English than on German, and specialist vocabulary does not make it easier. How big the gap is on your material only shows on your own audio. Do not expect perfection on the first run, judge instead whether the structure is right and the names roughly fit.

6. Measure the speed on your own material

The one number that counts is the ratio of computing time to recording length. If your machine needs twenty minutes for an hour of audio, that is perfectly fine for after-hours batch operation. If it needs two hours, you will stop doing it after the third time.

Measure it with a real file and not with a two-minute snippet, because the startup overhead distorts the value on short files. Note the result down together with model size and runtime.

Tip: If it is too slow, turn the runtime dial first and only then the model size. Switching from the original version to faster-whisper roughly keeps the accuracy at identical settings, going from small to tiny certainly costs some. Check the switch once on a file whose transcript you already know.

7. Improve the audio before you tinker with the model

The biggest gain in quality is rarely in the model. It is in the microphone. A clip-on microphone or a headset beats any model tier you were going to compensate with. Two people at a laptop microphone in a reverberant meeting room are difficult for every model, no matter how big.

The second lever is technical terms and proper names. Most environments let you supply a short text that primes the model on the vocabulary. Put the names of the participants and your ten most frequent technical words in there.

Tip: A model that guesses text out of bad audio invents plausible sentences. That is more dangerous than a gap, because it feels right while you read it. For important recordings the rule is: listen against the original at the decisive points.

8. Turn the transcript into something usable

A raw transcript is not a note yet, it is just more text. The second step belongs to a language model that turns the verbatim record into a summary, a task list and the decisions.

If you take this route for confidential conversations, that second step has to run locally too. Otherwise you have kept the recording in the building and then uploaded the full content anyway. Which text model fits your equipment is covered by Which local model fits your hardware.

Tip: Here lies the most honest limit of this workflow. Whisper does not separate speakers by itself, it delivers running text with no attribution of who said what. That speaker separation is exactly what the specialised cloud services do well. There are additional tools for it, but they are a project of their own and not part of an afternoon.

9. Sort out the legal side, it does not go away

A wrong conclusion is often drawn here. That the data does not leave your machine is a genuine and large advantage, because transmission to a third party does not happen and with it a whole stack of questions about data processing agreements and third-country transfers. But it does not automatically make the processing lawful.

You are still processing personal data and still need a legal basis and deletion periods, and you should check whether the processing belongs in your record of processing activities. For regular recordings and for the kinds of data from step 1 that is the normal case.

And then consent, where a shortcut is often taken. The sentence "I am recording this" at the start of the conversation is information, not consent. Whoever simply keeps talking afterwards has not agreed, they have merely not objected. The order that holds up: inform, ask explicitly, wait for a clear yes, note it down, and only then start the recording. If somebody says no, there is no recording, not even "just for me". In Germany this is not only a data protection question: recording the non-publicly spoken word without permission is a criminal offence under section 201 of the criminal code. Other countries have different rules, so check the ones where you are. A covert recording stays covert even if it only ever lands on your hard drive.

Tip: The clean separation between "does not leave the building" and "is therefore allowed" is set out at length in Local AI and the GDPR. For the wider frame, including the AI Act, take DACH legal, EU AI Act and GDPR.

10. Make it a routine, and check the two conditions

A workflow you reassemble every time dies after the second week. Set up a fixed folder the recordings go into, and a loop that runs the call from step 5 over everything in it and puts the text files next to them:

for f in ~/transcripts/inbox/*.wav; do
  whisper "$f" --model small --language German \
    --output_format txt --output_dir ~/transcripts/done
done

The start needs no more than that. The --output_format txt is not decoration: without it Whisper writes out every transcript in all formats, so five files per recording. With a different runtime you only swap the call in the middle.

And here comes the part that decides the whole promise. A model computing locally does not mean the file stays local.

The first condition is the folder. On Windows OneDrive backs up Desktop, Documents and Pictures on request, on macOS iCloud Drive stores Desktop and Documents. That setting is often already active without anybody having consciously set it. A recording that lands there goes along at the next sync. So put the working folder expressly outside the synchronised areas, and look in the settings of OneDrive, iCloud and Dropbox to see what is actually being backed up. While you are at it, think of the backup program running in the background.

The second condition is the program. That an interface has whisper.cpp inside does not mean it does nothing else. It can send usage data, check for updates or call a service for individual features. So disconnect the machine from the network once and let a recording run through. If a complete transcript comes out, you know that the transcription itself works without a connection. That is all you know: a program can also hold a transfer back and catch up the next time it connects, and the operating systems are explicitly built for that. If confidentiality really has to hold, block the program from outgoing connections permanently or keep the whole workflow disconnected. On Windows that means an outbound block rule in the firewall, because outbound traffic is allowed there by default.

Also keep the model name in a single place rather than scattered. Whisper has had several versions since launch, most recently large-v3 and turbo derived from it, and that development continues. Anyone who writes the name into five places stops switching out of sheer convenience.

Tip: Put the deletion period into the script right away. Recordings nobody needs any more are not an archive, they are a risk growing in the background.

What next

If you do not have a local model running at all yet, Your first local AI model in 30 minutes is the way in, and the lesson Does AI run on my machine explains the principle behind it. If you want to know whether your own machine pays off against the cloud at all, the calculation is in Local or cloud, the honest calculation. And if the local model is then to be connected to your own tools, MCP with local models takes it further.

Sources

Model sizes, memory requirements and licence details come from the official Whisper documentation, as of August 2026. The speed comparison is a measurement by the faster-whisper project for the small model on a Core i7-12700K with eight threads, it compares full precision against 8-bit compression and does not transfer one to one to other hardware. Check the current values at the source.

• Whisper, model overview and licence: https://github.com/openai/whisper
• Whisper, licence text with the condition on distribution: https://github.com/openai/whisper/blob/main/LICENSE
• Whisper, model card with version history: https://github.com/openai/whisper/blob/main/model-card.md
• whisper.cpp, implementation in C/C++: https://github.com/ggml-org/whisper.cpp
• faster-whisper, speed comparison: https://github.com/SYSTRAN/faster-whisper
• CTranslate2, accuracy under 8-bit compression: https://github.com/OpenNMT/CTranslate2/blob/master/docs/quantization.md
• Microsoft, folder backup of Desktop and Documents in OneDrive: https://support.microsoft.com/en-US/onedrive/back-up-your-folders-with-onedrive
• Apple, Desktop and Documents in iCloud Drive: https://support.apple.com/en-us/109344
• Section 201 of the German criminal code, violation of the confidentiality of the spoken word: https://www.gesetze-im-internet.de/englisch_stgb/englisch_stgb.html
• Article 9 GDPR, special categories of personal data: https://gdpr-info.eu/art-9-gdpr/
• Article 30 GDPR, records of processing activities including the exemption: https://gdpr-info.eu/art-30-gdpr/