Skip to content
the repo dude.
Menu
Saved

START WITH THE TASK

Run AI locally

Local AI includes different tasks: generating text and transcribing audio need different models and software. Choose your task first, then check memory, architecture and the model license. Local execution does not automatically mean every feature works offline.

Explore AI & language models

Where to start

For a local chat model, evaluate Ollama’s local-model path. For lower-level control over inference and backends, investigate llama.cpp. For audio transcription, start with Whisper.

Compare the building blocks

Documentation-based guidance, not hands-on tests. Source-check dates belong to each fact; publishing this page does not refresh them.

Repository and roleSetupExecution and costsLimitationsLicense and source
ollama/ollama

Get a local language model running quickly.

Install the native app/CLI or official Docker image, then select/download a model. Local service normally listens on port 11434. Cloud models require an Ollama account and can also be accessed directly through the cloud API.Checked 2026-11-20Execution requirements need review

MIT runtime; individual model weights retain their own licenses. Local use consumes owner hardware/storage. Reviewed cloud pricing offers per-token usage, Pro $20/month (or $200/year), Max $100/month and separate team/enterprise tiers; cloud usage is not free unlimited inference.Checked 2026-11-20

Hardware/backend support and usable context vary by model. Intel macOS lacks GPU acceleration; cloud inference runs remotely and current cloud docs exclude structured outputs. Cloud models can be retired; local and cloud capabilities are not identical.Checked 2026-11-20MIT

Original documentation

ggml-org/llama.cpp

A flexible engine for running language models.

Use official release binaries, Docker or a CMake source build with the desired backend. Supply a compatible GGUF model locally or through a model repository; other weight formats require supported conversion.Checked 2026-11-20Execution requirements need review

MIT inference code. Model licenses, downloads, compute, storage and optional hosted endpoints are separate; the runtime license does not grant rights to every supported model or make external hosting free.Checked 2026-11-20

Requires supported GGUF architectures and backend operations. Experimental tensor parallelism excludes several architectures and quantized KV cache; throughput depends on hardware/interconnect. Public serving needs appropriate authentication and deployment controls.Checked 2026-11-20MIT

Original documentation

openai/whisper

The speech model behind many transcription tools.

Install openai-whisper with Python/PyTorch and the system FFmpeg command-line tool. The README describes expected compatibility with Python 3.8–3.11 and recent PyTorch versions; Rust may be needed where a tokenizer wheel is unavailable. Model weights are loaded separately by the library.Checked 2026-11-20Execution requirements need review

The repository explicitly releases both Whisper code and model weights under MIT. Local inference still uses compute and storage; this license does not define pricing for a separate hosted transcription service.Checked 2026-11-20

The model card warns of hallucinated or repetitive text and uneven accuracy across languages, accents and dialects. It is not a real-time transcription system out of the box. Turbo is not trained for translation and returns the original language even when translation is requested.Checked 2026-11-20MIT

Original documentation

Compare these 3 repos →

Before you commit

Try one small input with a model that fits your machine. Verify where inference happens and whether model downloads, optional cloud features or plugins use a network service.

Read the detailed briefs

All task collections · Suggest a correction