START WITH THE TASK
Run AI locally
Local AI includes different tasks: generating text and transcribing audio need different models and software. Choose your task first, then check memory, architecture and the model license. Local execution does not automatically mean every feature works offline.
Where to start
For a local chat model, evaluate Ollama’s local-model path. For lower-level control over inference and backends, investigate llama.cpp. For audio transcription, start with Whisper.
Compare the building blocks
Documentation-based guidance, not hands-on tests. Source-check dates belong to each fact; publishing this page does not refresh them.
| Repository and role | Setup | Execution and costs | Limitations | License and source |
|---|---|---|---|---|
| ollama/ollama Get a local language model running quickly. | Install the native app/CLI or official Docker image, then select/download a model. Local service normally listens on port 11434. Cloud models require an Ollama account and can also be accessed directly through the cloud API.Checked 2026-11-20 | Execution requirements need review MIT runtime; individual model weights retain their own licenses. Local use consumes owner hardware/storage. Reviewed cloud pricing offers per-token usage, Pro $20/month (or $200/year), Max $100/month and separate team/enterprise tiers; cloud usage is not free unlimited inference.Checked 2026-11-20 | Hardware/backend support and usable context vary by model. Intel macOS lacks GPU acceleration; cloud inference runs remotely and current cloud docs exclude structured outputs. Cloud models can be retired; local and cloud capabilities are not identical.Checked 2026-11-20 | MIT |
| ggml-org/llama.cpp A flexible engine for running language models. | Use official release binaries, Docker or a CMake source build with the desired backend. Supply a compatible GGUF model locally or through a model repository; other weight formats require supported conversion.Checked 2026-11-20 | Execution requirements need review MIT inference code. Model licenses, downloads, compute, storage and optional hosted endpoints are separate; the runtime license does not grant rights to every supported model or make external hosting free.Checked 2026-11-20 | Requires supported GGUF architectures and backend operations. Experimental tensor parallelism excludes several architectures and quantized KV cache; throughput depends on hardware/interconnect. Public serving needs appropriate authentication and deployment controls.Checked 2026-11-20 | MIT |
| openai/whisper The speech model behind many transcription tools. | Install openai-whisper with Python/PyTorch and the system FFmpeg command-line tool. The README describes expected compatibility with Python 3.8–3.11 and recent PyTorch versions; Rust may be needed where a tokenizer wheel is unavailable. Model weights are loaded separately by the library.Checked 2026-11-20 | Execution requirements need review The repository explicitly releases both Whisper code and model weights under MIT. Local inference still uses compute and storage; this license does not define pricing for a separate hosted transcription service.Checked 2026-11-20 | The model card warns of hallucinated or repetitive text and uneven accuracy across languages, accents and dialects. It is not a real-time transcription system out of the box. Turbo is not trained for translation and returns the original language even when translation is requested.Checked 2026-11-20 | MIT |
Before you commit
Try one small input with a model that fits your machine. Verify where inference happens and whether model downloads, optional cloud features or plugins use a network service.