How to run Whisper yourself, and when it is worth it

What Whisper actually is

Whisper is a family of speech recognition models that OpenAI trained on a large amount of audio paired with text, then released publicly, weights and all, in September 2022. "Open-source" here means something specific: you can download the trained model files and run them on your own hardware, for free, with no API key and no per-minute charge. That is unusual. Most speech recognition before Whisper lived behind a paid API from a handful of large vendors.

Whisper changed transcription because it was good, not just because it was free. It handles a wide range of accents, background noise, and languages (around 99 of them, with wildly varying quality) noticeably better than the open models that came before it, and it does translation from many languages into English as a side effect of how it was trained. Within a year of release it became the base layer under a large share of the transcription tools on the market, commercial and hobbyist alike.

It comes in several sizes, from tiny (fast, rough) up through base, small, medium, to large (slow, accurate), and OpenAI and others have released newer variants since, including turbo models tuned to run faster with only a small accuracy cost. The size you pick trades speed for accuracy, and that trade-off runs through everything else in this guide.

One thing worth saying plainly: Whisper is a genuinely good piece of engineering, and nothing below is about talking you out of using it. It has real limits, covered later, but describing it fairly matters more than selling you on or off it.

The hardware picture, honestly

Whisper runs on a plain CPU. No special hardware is required, and for short files that is fine. But CPU transcription is slow, often close to real time or slower for the larger, more accurate models, meaning an hour of audio can take an hour or more to process. For a five-minute voice memo, you will not notice. For a backlog of hour-long interviews, you will feel every minute of it.

A GPU changes the picture completely. Whisper's underlying architecture is built for the kind of parallel math a graphics card does well, and on a decent GPU the same hour of audio that took an hour on CPU might take a few minutes. If you already own a gaming PC or a workstation with an NVIDIA card, you likely have everything you need to run the large, most accurate model at a comfortable speed. If you do not own one, buying a GPU purely for occasional transcription is rarely worth it on its own.

Apple Silicon Macs occupy a genuinely good middle ground. The M-series chips share memory between CPU and GPU and have dedicated matrix hardware, and tools built for them (whisper.cpp in particular has strong Apple Silicon support) get meaningfully better speed than a CPU-only run without needing a separate graphics card at all. If your daily machine is an M1 through M4 Mac, you are closer to the GPU experience than the CPU one, and that is probably the most common "it just works" setup people report.

The realistic summary: CPU works and costs nothing extra, but budget real wall-clock time for anything beyond short clips. A GPU or an Apple Silicon Mac turns Whisper from something you run overnight into something you run while you make coffee.

Want to try it now? Upload a file to FastScribe. Your first one is free, no signup.

The main routes people take

whisper.cpp is the most widely used way to run Whisper locally without Python or a GPU driver stack. It is a from-scratch C++ reimplementation of Whisper, built to run efficiently on ordinary CPUs and on Apple Silicon, and it ships as a small, dependency-light program you compile once and then run from the command line. If you want the simplest path to "one program, no cloud, works offline," this is usually it.

faster-whisper is the route most people reach for when they have a GPU and are comfortable with Python. It reimplements Whisper's inference on top of CTranslate2, a library built for fast, memory-efficient transformer inference, and in practice it runs several times faster than the original OpenAI implementation on the same hardware, with lower memory use. It is the common choice behind more technical, scriptable, or batch-processing setups.

A third category worth knowing about is desktop wrapper apps: point-and-click programs that bundle whisper.cpp or a similar backend behind a normal window with drag-and-drop, a model picker, and export buttons. Several exist across Mac, Windows, and Linux, free and paid, open-source and not. They trade a little transparency and control for not touching a terminal at all, and for many people that trade is the right one. This guide focuses on the command-line tools underneath, since understanding those makes any wrapper app easier to reason about.

None of these routes involves a hidden trick or a paid unlock. They are different levels of the same stack: a model file, some code that feeds audio into it, and something that writes out the result.

A walkthrough, at the conceptual level

Whichever tool you pick, the shape of the job is the same. First, you get the tool itself, for whisper.cpp that means compiling a small C++ project (a couple of commands, no separate install of anything heavy), for faster-whisper it means a Python environment and a package install. Second, you get a model file: a download, typically a few hundred megabytes to a couple of gigabytes depending on which size you chose, that the tool loads at run time. You choose this once and reuse it for every file after.

Third, you convert your audio to the format the tool expects. Whisper was trained on 16kHz mono audio, and while some tools handle conversion internally, the reliable move is to do it yourself with ffmpeg, the free, universal audio and video converter almost every guide like this leans on. A conversion command looks roughly like this: ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav That takes whatever you recorded, an mp3, a video's audio track, a voice memo, and produces a clean 16kHz mono WAV file, the shape Whisper wants.

Fourth, you run the tool against that file, pointing it at the model you downloaded. With whisper.cpp, a basic invocation looks something like: ./main -m models/ggml-base.en.bin -f output.wav That single command loads the model, processes the WAV file, and prints the transcript. Fifth, you choose an output format, most tools can write plain text, or a timestamped subtitle format like SRT or VTT, as a command-line flag, so you decide up front whether you want prose or timed captions.

That is genuinely the whole shape of it: get the tool, get a model, convert the audio, run it, pick a format. The details differ between whisper.cpp and faster-whisper, and specific flags change between versions, so treat the commands above as illustrative rather than something to copy verbatim without checking your tool's current documentation, but the five-step shape does not change.

The gotchas nobody mentions

The size-versus-accuracy-versus-speed trade-off is bigger than it sounds. The tiny and base models are fast but make real mistakes, wrong words, dropped phrases, on anything but very clear audio. The large model is meaningfully more accurate but can be ten times slower or more on the same hardware. There is no single right answer, only a decision you make per job: quick and rough for a note to yourself, large and slow for something you will quote.

Long files are their own problem. Whisper processes audio in chunks internally, and very long recordings, a two-hour lecture, a long podcast, can run into memory pressure or slow down disproportionately compared with a short clip, especially on CPU or a GPU with limited memory. Splitting a long file into smaller pieces before transcribing, and stitching the outputs back together, is a common and unglamorous workaround.

Memory matters more than people expect going in. The larger models need real RAM (or VRAM, on GPU) to load, and running out partway through a file produces a crash or a silent failure rather than a graceful slowdown. Check the model's stated memory requirement against your machine before you commit to a large batch job.

Two behaviors surprise almost everyone the first time. Whisper can hallucinate on silence or near-silence, confidently producing sentences, sometimes whole paragraphs, of text where nothing was actually said, most often at the start or end of a file or during long quiet stretches. And Whisper has no built-in concept of who is speaking: it produces one stream of text with no speaker labels, so a two-person interview comes back as an unbroken block of prose you would need a separate speaker-diarization tool, run as an extra step, to split apart.

When DIY wins, and when a service wins

DIY genuinely wins in a few clear situations. If you transcribe a large, ongoing volume of audio, hours a week, every week, the cost of a per-minute or subscription service adds up in a way a one-time setup does not. If your material is genuinely sensitive and you want it to never leave a machine you control, air-gapped privacy is a real and legitimate reason to run everything locally, no upload, no third party, full stop. And if you like tinkering, comparing model sizes, testing settings, watching a command-line tool work, that is a real form of value too, not everyone wants their tools to be invisible.

A service wins just as clearly in the opposite cases. If you transcribe occasionally, a handful of files a month, the hour or more you would spend once getting a local setup working, picking a tool, downloading a model, learning the flags, usually costs more than the service fee it was meant to avoid. Setup time is a real cost even when the software itself is free. A service also hands you the parts DIY leaves as homework: exports in multiple formats without extra tooling, a history of past transcripts you can search and return to, and nothing to keep updated as tools change.

It is worth being honest about what actually differs between running Whisper yourself and using an automated transcription service: usually not the underlying technology. FastScribe runs this same class of open speech recognition models, on its own servers, rather than sending your audio to a third-party AI service. The real choice is not secret technology versus DIY, it is whose time and whose machine does the work. Running it yourself spends your time and your hardware; using a service spends a few dollars and none of either.

Neither answer is universally right, and the two are not mutually exclusive. Plenty of people run Whisper locally for routine, high-volume material and reach for a service for the occasional file they need fast, formatted, and off their plate. Knowing how the local route actually works, which this guide just walked through, is what lets you make that call for real instead of guessing.

Where FastScribe fits

FastScribe runs this same class of open speech recognition models, on its own servers, so the honest way to think about the choice is not secret technology versus DIY, it is whose time and whose machine does the work. If you already have a capable GPU or an Apple Silicon Mac, transcribe often, and enjoy the setup, running Whisper yourself is a real, free, and worthwhile option, and this guide was written to actually get you there rather than talk you out of it. Where FastScribe fits is the case where that setup cost is not worth paying: an occasional file, a deadline today, no interest in comparing model sizes. Your first file transcribes free with no signup, up to 50 MB and 10 minutes; a free account raises that to 5 files a day up to 30 minutes each; Pro is $12 a month for files up to 5 hours. Audio is deleted the moment your transcript is ready, and exports cover TXT, SRT, and VTT, with DOCX on Pro. Try both, honestly, on the same file, and let your own volume and hardware decide.