How to run an LLM locally: Ollama, LM Studio, step by step

Running a large language model on your own machine used to be a weekend project: Python environments, GPU drivers, half-understood quantization jargon. Today it is closer to installing an app. You type a prompt, a model that lives on your disk answers, and nothing you wrote ever left the building. This guide walks the whole path in one sitting — install a runner, pull your first model, then put a real interface on top — using tools that are free, established, and documented well enough to recover from missteps.

Why go local at all? Four reasons come up again and again. Privacy: once a model is downloaded, your prompts and documents are processed on your machine, which matters for drafts you would never paste into a hosted chat. Offline: after the download, chat works on a flight, on a train, or in the middle of an outage. No meters: there is no per-token bill and no monthly quota to watch. Availability: no rate limits and no capacity windows. One expectation to set before anything else — small local models are not frontier models. They lose reasoning depth on hard problems, and quantized downloads trade quality for memory. That trade is the price of the four advantages above.

01What you need before you start

• An operating system the tools support. Ollama runs on Windows, macOS and Linux. LM Studio ships for macOS (Apple Silicon), Windows (x64/ARM64) and Linux (x64). Open WebUI deploys as a single Docker container, so any machine that runs Docker qualifies.
• Memory in the right tier. With 8 GB of RAM, small 7–8B models run reasonably. A 16 GB machine is a comfortable floor for mid-size models, and larger ones want real VRAM on a discrete GPU. Each model's download page states its memory footprint, so check it before you pull.
• Disk space. Local models are multi-gigabyte files, and it is easy to collect several while comparing. Budget for cleanup, not just download.
• Command-line tolerance, in varying doses. The runner path means typing commands. The graphical path means none. The runner comes first here because the interfaces build on it.

02Step 1 · Install Ollama, the runner

Ollama is the engine this whole stack sits on. It is free, MIT-licensed, and its pitch is hard to argue with: install it, type one command, and a model answers in your terminal. It also starts a local server in the background, with an OpenAI-compatible API, so scripts and other apps can use the same model. It is rated ★ 4.4 on its object page.

macOS / Linux:

curl -fsSL https://ollama.com/install.sh | sh

On macOS you can alternatively download the app from ollama.com.

Windows (PowerShell):

irm https://ollama.com/install.ps1 | iex

The ollama.com installer does the same job for click-first people.

Confirm the install with ollama --version.

The pitfall you will hit here: installing Ollama does not give you a chat window. Ollama is a runtime, not a platform — user management, usage dashboards and scaling are not its features. If you double-click an icon and expect a ChatGPT-style app, you will be confused for a minute. That is normal. The chat experience arrives in steps 2 and 3.

03Step 2 · Pull a model and have your first conversation

One command downloads a model and starts chatting — identical on macOS, Linux and Windows:

ollama run gemma4

The first run shows a download progress bar, then drops you into a chat prompt in your terminal. Type a message, get an answer, and type /bye to leave. The catalog goes well beyond this one model — families like Llama, Qwen, Gemma and DeepSeek are a download away, each listing tagged sizes so you can pick what fits your memory. The full library is at ollama.com/library.

How to pick your first model: if your machine has 8 GB of memory, start with the smallest variants and watch the speed. Small models keep responses usable; the 7–8B class is the usual starting band, and you size up only if the pace stays tolerable. If the model you pulled crawls, the honest diagnosis is hardware, not the software — on weak machines the local experience is patience, not productivity. Even a 64 GB machine cannot comfortably load a 405-billion-parameter model, so do not chase the biggest catalog entry on the list.

The pitfall you will hit here: the first answers will feel slower and shallower than the cloud chat you are used to. That is the small-model tradeoff described above: local models shine at drafting, brainstorming and sensitive text, not hard reasoning problems.

Two housekeeping commands you will actually use, the same on all three systems:

ollama list
ollama rm gemma4

ollama list shows what you have downloaded; ollama rm deletes a model you are done with. Disk fills up fast once you compare models.

04Step 3 · Add an interface: LM Studio or Open WebUI

With a runner installed and a model pulled, you have a working but spartan setup. The next step is a face on top of it. Two directions cover most people: a desktop application that needs no terminal at all, and a browser interface built for sharing.

Option A · LM Studio, the graphical route

LM Studio is a desktop application for running local models: search and download models from Hugging Face inside the app, chat with them fully offline, and flip on an OpenAI-compatible local server when your own code needs an endpoint. The app detects your hardware and recommends models that will actually fit, offers per-layer GPU offload, and does document question-answering without anything leaving the machine. It runs on macOS (Apple Silicon), Windows and Linux, is free to download and run on your own hardware, and is rated ★ 4.5 on its object page. Install it by downloading the installer from lmstudio.ai for your platform.

Choose LM Studio if clicking beats typing for you. It is the least intimidating door into local models — no terminal, no Docker, no accounts — and reviewers consistently frame it as the default recommendation for non-developers.

The pitfall you will hit here: the app scales its suggestions to your hardware, but it cannot repeal physics. Mid-size models want roughly sixteen gigabytes of memory, larger ones want real VRAM, and older laptops crawl where demo machines fly. Trust the app's hardware-aware recommendations over your optimism. Also worth knowing: LM Studio is proprietary freeware around open-weight models, and if your workflows later become scripts and schedules, you will likely graduate to Ollama or raw llama.cpp — the graphical muscle memory does not transfer to automation.

Option B · Open WebUI, the browser route

Open WebUI is a self-hosted web interface that speaks to the Ollama server you installed in step 1, plus any OpenAI-compatible endpoint. It adds the things a terminal will never give you: multi-user accounts, groups with per-group model access, document uploads for chat-with-your-files, voice input, and side-by-side comparison of two models answering the same prompt. It is open source and deploys as a single Docker container, and it is rated ★ 4.5 on its object page.

With Docker installed, one command brings it up (macOS, Linux, or Windows with Docker Desktop):

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Read the flags before you paste: -p 3000:8080 maps port 8080 inside the container to port 3000 on your machine, and --add-host=host.docker.internal:host-gateway lets the container reach the Ollama server running on your host — inside a container, plain 127.0.0.1 points at the container itself, which is why the docs call this flag out for exactly this setup.

The pitfall you will hit here: after the container starts, open http://localhost:3000 — not port 8080. The 8080 in the command is the port inside the container; 3000 is the address on your machine. Visiting 8080 in a browser gets you nothing and convinces people the install failed.

How to choose between them

Pick LM Studio if you are one person who wants a polished desktop app with zero setup ceremony. Pick Open WebUI if you want browser access from any device in the house, multiple users with their own accounts and limits, or a machine that stays on and serves the household. The two are not rivals for the same slot: LM Studio replaces the terminal, Open WebUI sits in front of the runner you already installed, and you can run both.

05Three variations worth knowing

If memory or VRAM runs short. Drop a size class rather than fighting for it: smaller models run where larger ones refuse to, and quantized downloads trade some quality for memory. LM Studio's hardware-aware suggestions and per-layer GPU offload exist precisely for this. On a machine with integrated graphics only, run the smallest models and keep expectations modest. The failure mode to avoid is downloading the flagship-size model first and concluding that local AI is slow.

When you want to try other models. The terminal path pays off here: ollama pull fetches another family, ollama list shows your inventory, and ollama rm cleans up. Ollama's Modelfile format lets you pin a version and set a system prompt, so a model plus its settings travels as one file. In Open WebUI you just switch models in the chat and compare answers side by side — reviewers commonly run several models against one prompt to judge them. One caution: the catalog moves faster than its own entries, so check dates before building on any specific model.

When you want it on your phone or in another room. The browser route is the local-network route. Open WebUI is a web server, so other devices on your home network can reach the machine running it, and per-user accounts let a family or small team share one interface without sharing one login. Ollama can also be reached over the network via its documented OLLAMA_HOST setting, but keeping the runner bound to the machine and putting Open WebUI — with its accounts and permissions — in front is the safer shape. Keep exposure to your own network; a public model server is a different project with different homework.

06Which one is yours

The honest end state for most people is a stack, not a choice: Ollama as the engine, one of the two interfaces above as the face, and a model sized to the machine you actually own. Start with the smallest model that answers, learn the housekeeping commands, and size up only when the use case demands it.

07Frequently asked questions

Do I need a GPU to run a local LLM? No. With 8 GB of memory, small 7–8B models run reasonably well on CPU alone. A discrete GPU improves speed and unlocks larger models, and mid-size models are most comfortable around 16 GB of RAM. Check the memory footprint on each model's download page before you pull it.

Is my data actually staying on my machine? Once a model is downloaded, your prompts and the model's answers are processed locally — that is the core promise of all three tools, and LM Studio extends it to document analysis and voice transcription by default. The nuance worth knowing: the clients do use the internet to fetch models and updates, and LM Studio's optional paid tiers add hosted inference of open models on US infrastructure under a Zero Data Retention policy. Audit what matters for your use.

Ollama or LM Studio — which should I start with? Both run llama.cpp under the hood, and both give you the same open model families, so trying both costs little but download time. If you want a graphical app with a model browser, start with LM Studio; if you live in a terminal or want an API for scripts, start with Ollama. Many people start with the GUI and automate with Ollama later — that progression is the common story.

What does all this cost? The software is free at every layer of this guide. Ollama is MIT-licensed with free model downloads, LM Studio is free to download and run on your own hardware, and Open WebUI is open source and free to self-host. Your real costs are hardware if you run models locally, or the API charges of whatever cloud provider you connect to Open WebUI. The paid tiers that exist add convenience and limits, not access to the basics.