How to Host and Run Your Own Local LLM at Home: What It Actually Takes, How Much Memory Buys Which Model, the Machine Menu Priced This Week, Ollama vs LM Studio vs Jan, the First Run Step by Step, Serving the Whole House Safely, and What It Costs Against a Subscription

This guide is the whole thing from zero: what a local setup actually is, how much memory buys which model, what the machine options cost this week, which of the five main apps to install, the exact commands for a first run, how to serve your phone and your family without serving the whole internet, and what it costs in electricity against a subscription. Every number is from a primary source checked on November 3, 2026: Ollama's model library and FAQ, LM Studio's docs, Apple's and NVIDIA's spec pages, price trackers, the EIA, Cisco Talos, and Hugging Face model cards. Where a figure is a vendor claim or a community measurement, it says so. The diagrams are ours.

The short version

• Who should do this: anyone whose data can't leave the machine, anyone paying for two or more AI subscriptions, anyone who needs offline, and anyone who wants coding agents that run on their own hardware. If you'd use it an hour a week, a $20 subscription is cheaper for the first four years (the maths is at the end).
• Memory is the whole decision. A Q4-quantized model needs about 0.6GB per billion parameters, plus room for the conversation. 16GB runs good small models (Gemma 4 12B, Qwen3.5 9B, gpt-oss-20b); 24–32GB runs every model on our open-source ranking, including the best one, Qwen3.8-27B at 18GB.
• Apple Silicon is the easiest path, because the memory is unified and the apps use Apple's MLX engine. The $899 Mac mini M6 (16GB) is the cheapest good start; the 32GB build at $1,299 is the smart one. A used RTX 3090 is still the GPU-side answer, but it now sells for $1,300–1,900, not the $700 of a year ago.
• Pick Ollama if other tools will talk to the model, LM Studio if you want a graphical app. Both are free, both run on Mac, Windows and Linux, both serve an OpenAI-style API so coding tools and chat apps plug in. Jan is the fully open-source alternative; llama.cpp is the engine underneath all three.
• The first run takes 5 to 30 minutes, almost all of it download. The two settings beginners trip on: the default context window (Ollama 4,096 tokens, LM Studio similar) and, in LM Studio, GPU offload controls hidden behind Developer mode.
• Don't port-forward it. Ollama's API has no password. Cisco Talos found 1,139 exposed servers on the public internet in ten minutes of scanning; SentinelLABS counted 175,108 over ten months. Use your LAN or Tailscale.
• Quality gap, honestly: the best model you can run on a 24GB machine scores 20 on Artificial Analysis's Intelligence Index; the open frontier (DeepSeek V4.1 Flash, GLM-5.3) scores 32 and needs hundreds of gigabytes; the closed frontier (Gemini 4 Carbon, Opus 5.5) scores 58–59. Local is for privacy, cost and control, not for beating ChatGPT.

What you get, and what you give up

Three things pull people to local models, and the posts we read this fortnight repeat them in the same order: privacy, no ongoing bill, and offline use. The privacy claim is the cleanest. Ollama's FAQ says it plainly: 'Ollama runs locally. We don't see your prompts or data when you run locally'. LM Studio's docs: 'Nothing you enter into LM Studio when chatting with LLMs leaves your device', with the only network calls being model search, downloads and update checks. The fourth motive, newer this year, is agents: Ollama's `ollama launch` (January 2026) sets up Claude Code, OpenCode or Codex against a local model in one command, so the coding assistant reads your repository without the repository leaving your disk.

What you give up is intelligence at the top end, and nobody selling hardware says this clearly enough. On Artificial Analysis's open-weights leaderboard the best model that fits a 24GB machine, Qwen3.8-27B, scores 20 on the Intelligence Index. The best open models score 32 (DeepSeek V4.1 Flash, GLM-5.3), and they are 155GB to 467GB downloads. The closed frontier models you're used to score 58–59 on the same index. A good local model in 2026 is roughly where the paid chatbots were about eighteen months ago: excellent for drafting, summarising, coding help, structured extraction, and anything with your private documents; noticeably weaker on hard reasoning, obscure facts, and long agentic tasks. Most people who set one up end up using both, local for the private and repetitive, cloud for the hard.

There is also a middle path worth knowing before you spend anything: Ollama's cloud tags run the datacenter-sized open models (Mistral Large 4, DeepSeek V4.1 Flash) on Ollama's servers through the same command line, so you can try the 1-trillion-parameter tier without owning it. It is not local, and Ollama says so; you can switch the whole cloud side off with `OLLAMA_NO_CLOUD=1` if you want a machine that provably never phones out.

Step zero: how much memory do you have?

Before tools, before models, look up one number: how much memory your GPU has (VRAM, on a PC with a graphics card) or how much unified memory your Mac has. On a PC without a discrete GPU, the answer is effectively 'system RAM, slowly', and we'll show why in a moment. The model's weights have to fit in that memory, whole, with room left for the conversation. Everything else in this guide is downstream of that number.

The rule that generates that ladder: a model quantized to 4 bits needs roughly 0.6GB per billion parameters. Ollama's own download sizes are the receipts: Llama 3.1 8B is 4.9GB, Qwen3 32B is 20GB, Llama 3.3 70B is 43GB. Add a couple of gigabytes for the runtime and the operating system, then add the conversation itself, which is the part everyone forgets and we'll come back to.

'Q4_K_M', the suffix on most downloads, is the quantization: how many bits each weight is stored in, after the model was trained at 16. Qwen3.8-27B exists on Ollama as bf16 at 56GB, q8_0 at 30GB and q4_K_M at 18GB, and the 18GB one is what `ollama run qwen3.8` fetches. The quality cost at Q4 is real but small for chat and coding; the community consensus, which matches our testing for the ranking, is that going below Q4 to squeeze a model into memory usually loses more than stepping down to a smaller model at Q4. One exception: gpt-oss-20b ships natively in OpenAI's MXFP4 format at 4.25 bits and should not be re-quantized at all.

One more number for Mac buyers: macOS holds back a share of unified memory from the GPU, around a quarter on larger machines by community reports (Apple doesn't document the figure, and it can be raised). A 64GB Mac behaves like roughly a 48GB GPU for model purposes. Windows machines with the AMD Ryzen AI Max chips have a similar split: Framework says up to 96GB of 128GB can be dedicated to the GPU.

If you're buying: the machine menu, priced this week

The advice in nearly every post we read is the right one: start on the machine you already own. If it has 16GB, it runs real models today. If you are buying, the choice comes down to two numbers, memory capacity (which model fits) and memory bandwidth (how fast it answers), and the diagram below is the one that explains why a $899 Mac can feel faster than a $2,000 gaming PC whose model doesn't fit in VRAM.

The arithmetic behind that chart is the only piece of theory you need. To produce one token, the chip reads every weight once, so tokens per second can never exceed memory bandwidth divided by model size (a clear write-up, with the same formula NVIDIA engineers use). An 18GB model on a desktop CPU's 96GB/s dual-channel DDR5 tops out around 5 tokens per second, slower than you read. The same model on a Mac Studio's 460GB/s is capped near 25; on an RTX 5090's 1,792GB/s near 100. This is also why mixture-of-experts models like Gemma 4 26B and GLM-4.7-Flash feel fast: they read only the 'active' 3–4 billion parameters per token while occupying the full 18–19GB on disk and in memory.

Three things changed since most 'build a local AI box' guides were written. Used GPU prices roughly doubled: the RTX 3090, the long-time budget pick, averages $1,573 across 328 listings this week, and the RTX 4090 averages $3,471. The RTX 5090 never came down: list $1,999, street about $6,600, with no 50-series Super refresh shipped. And NVIDIA's DGX Spark went from $3,999 to $4,699 in February, citing memory costs, to $6,950 now, with a 64GB version at $4,999 since October 23. Against that, Apple's prices held: the Mac mini M6 at $899 and the Mac Studio M5 Max at $2,499 are the same money as the previous generation, with more bandwidth. That, more than any software reason, is why 'just get a Mac' is the consensus in the posts this month.

If you already own a gaming PC with a 12GB or 16GB card: it's a fine machine for the 8–14B class and for gpt-oss-20b, and it will be fast. Just know the 27B models won't fit, and when a model doesn't fit, the runtime quietly spills the remainder into system RAM and the speed falls off a cliff. Ollama's `ollama ps` shows the split as a percentage; if it says anything other than 100% GPU, pick a smaller model or a lower quant.

Pick the app: Ollama, LM Studio, Jan, llama.cpp, Open WebUI

Ollama: the default

Ollama is the one most posts assume. It's a background service plus a command line (and since July 2025 a desktop chat app on Mac and Windows), it installs with one line, and it exposes an API on port 11434 that practically every AI tool knows how to talk to. Version 0.40.14 shipped today; the project has about 186,000 GitHub stars. On Apple Silicon it can use Apple's MLX engine, in preview since March and recommended for Macs with more than 32GB. Its defaults are conservative: the context window is 4,096 tokens unless you raise it, which is the first thing to change once you're running.

LM Studio: the graphical one

LM Studio is a full desktop application: a model browser that searches Hugging Face, a chat window, a document-chat mode, and sliders for the engine. It runs llama.cpp everywhere and MLX on Apple Silicon, supports Apple Silicon Macs (M1 and later, not Intel), Windows x64 with AVX2 and Windows on Arm, and Linux, and recommends 16GB of RAM, saying 8GB 'may work' with small models. It has been free at home and at work since July 2025; the current release is 0.4.28 from October 31. It also runs an OpenAI-compatible server on port 1234 and ships a command line, `lms`, for headless use.

The beginner complaints about LM Studio are specific and they're real. The context length defaults to a few thousand tokens and has to be changed in two places, once for the loaded model and once as the default for next time; a Japanese user laid out the exact steps in April. And the GPU-offload slider that decides whether your graphics card is used at all is hidden unless Developer mode is on; LM Studio's own account told a confused user in January: 'Still there! Be sure to enable Developer Mode. Settings > Developer > flip it on.' If LM Studio feels slow on a machine with a good GPU, that switch is the first place to look. Two open bug reports from September and October describe slowdowns on very long chats.

Jan: the fully open one

Jan is a desktop app in the LM Studio mould that is itself open source under Apache 2.0, about 45,000 stars, version 0.8.8 on October 28. It bundles llama.cpp, runs on Windows 10+, macOS 13.6+ and Linux, serves an OpenAI-compatible API on port 1337, and has built-in MCP connectors so the model can use tools. Its README's memory guidance is a useful sanity check from a third vendor: 8GB for 3B models, 16GB for 7B, 32GB for 13B.

llama.cpp: the engine

Everything above runs on llama.cpp or Apple's MLX underneath. Using it directly gets you every flag and a new build several times a day (b12076 today). Its server has a built-in web UI and, as of the current README, a unified command: `llama serve -hf <huggingface-repo>` downloads and serves a model in one go, defaulting to the Q4_K_M quant and port 9931. The GGUF file format that every local model ships in is llama.cpp's own.

Open WebUI: the household front end

Open WebUI is not a runtime; it's a ChatGPT-style web page with user accounts, chat history and document upload that sits in front of Ollama (or any OpenAI-style server). One Docker command, about 158,000 stars, and the thing you want if more than one person in the house will use the box. Note its license changed in April 2025 to a custom one that requires keeping the Open WebUI branding unless you're under 50 users; irrelevant at home, relevant if you're deploying it at work.

The first run, step by step

Ollama (Mac, Linux, Windows)

Mac and Linux share one install line from ollama.com/download; Windows has its own, or an installer. Then `ollama run` with a model name downloads it if needed and drops you into a chat. Our recommendation per memory tier is in the next section; this uses the best 16GB-class model.

curl -fsSL https://ollama.com/install.sh | sh
# Windows (PowerShell):  irm https://ollama.com/install.ps1 | iex

ollama run gemma4:12b          # 8GB download, fits 16GB machines
ollama run qwen3.8             # 18GB, the best model for 24GB+

ollama ps                      # is it 100% on the GPU?
ollama list                    # what's downloaded

Two settings to know on day one. `OLLAMA_CONTEXT_LENGTH=32768` (as an environment variable, or in the desktop app's settings) lifts the 4,096-token default so the model remembers a long conversation or a whole document; the memory cost is in the next diagram. And `ollama launch claude` or `ollama launch codex` wires a coding agent to the local model, which is the thing a lot of the posts this month were actually trying to do.

LM Studio

Download the app for your platform, open it, and use the search icon to find a model; the listing shows the download size and whether it will fit your memory before you click. Load it, chat. Then, before you judge the speed: Settings → Developer → turn Developer mode on, open the model's load settings, and confirm GPU offload is at maximum; and raise the context length in both the current-session and default fields. If you'd rather not click, the bundled CLI does the same:

lms server start
lms load qwen3.8-27b --gpu=max --context-length 32768
# OpenAI-compatible API is now at http://localhost:1234/v1

llama.cpp directly

# any GGUF repo on Hugging Face; quant defaults to Q4_K_M
llama serve -hf ggml-org/GLM-4.7-Flash-GGUF:Q4_K_M
# web UI and OpenAI-style API at http://localhost:9931

Apple Silicon, the bare-metal way

pip install mlx-lm
mlx_lm.chat

MLX LM is Apple's own Python package for running models on Apple Silicon; LM Studio uses it under the hood and Ollama is adopting it. You don't need to touch it to get started, but it is why Macs are the easiest path: the memory is one pool, there is no 'offload' to configure, and the vendor writes the engine.

Which model to run

Our open-source models ranking tests the field on real hardware and re-checks sizes monthly; here is the short mapping from memory to model, with sizes from Ollama's library today. The ranked five are linked to full reviews with install commands and machine recommendations.

Two corrections to common advice. 'Small DeepSeek models that fit in a few gigabytes' means the R1 distillations from January 2025 (1.1GB to 9GB); every 2026 DeepSeek release is datacenter-sized, with V4 Flash's smallest sensible GGUF at 155GB. And 'the new Mistral Large 4' is real, weights released October 29, but it is a 1-trillion-parameter model whose repository is 1.06TB under a modified-MIT 'Mistral Ecosystem' license; on Ollama it exists only as a cloud tag. It belongs in the next table, not on your Mac.

Context: the setting everyone changes first, and what it costs

Every token in the conversation leaves a trace in memory called the KV cache, and it grows linearly with context length. Together AI's write-up gives the formula and a worked number: Llama 3.1 8B stores 0.125 MiB per token, so a full 131,072-token context costs about 16GB, more than the model's own weights at 16-bit precision and more than three times its 4.9GB Q4 download. Larger models store more per token (the 70B is 0.3125 MiB, about 40GB at 128K). This is why Ollama defaults to 4,096 tokens and LM Studio to a similar figure, why 'I set the context to maximum and it got slow' is a recurring post, and why the honest purchase rule is to buy memory for the context you'll actually use. A 32K window, enough for a long document or a day's chat, costs 4GB on an 8B model and is the setting we'd start with. Runtimes can store the cache at 8-bit to halve this; the apps expose it as 'KV cache quantization'.

Serving the house without serving the internet

A model on a Mac mini in a closet is more useful than one on your laptop: it's always on, it idles at 4 watts, and everyone's phone can use it. By default Ollama listens only on 127.0.0.1, meaning only programs on that machine can reach it. Setting `OLLAMA_HOST=0.0.0.0` makes it answer on your Wi-Fi, which is what you want for a house, and it is also the exact setting people copy from a forum and then forward through their router. Ollama's API has no authentication: anyone who can reach the port can run prompts, pull models and see what's loaded. Cisco Talos scanned the public internet in September 2025 and found 1,139 exposed Ollama servers, over a thousand of them in the first ten minutes, about a fifth actively hosting models. SentinelLABS and Censys then counted 175,108 unique exposed hosts across 130 countries over ten months of scanning.

The safe pattern costs nothing. Keep the server on your LAN, or leave it on localhost and reach it through Tailscale, the free mesh VPN, from your phone anywhere; Tailscale's own how-to walks through exactly this with Ollama and Open WebUI. For the shared chat page itself, Open WebUI's one-line Docker command finds Ollama on the same machine automatically:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main
# then open http://<server>:3000 and create the first (admin) account

What it costs, against a subscription

Electricity is the whole running cost, and it's small. A Mac mini M6 at 4 watts idle and 70 watts flat out, worked two hours a day, uses about 7 kWh a month, which is about $1.25 at the EIA's August 2026 average of 18.16¢/kWh. A Mac Studio M5 Max is 7 watts idle and 200 at maximum, so a few dollars. A gaming PC with an RTX 5090 is the outlier: 575 watts for the card alone, so eight hours of daily inference lands near $30 a month in electricity.

Against that, the hardware. A $899 Mac mini takes about four years to pay back against a single $20 subscription, and about two years against two of them. That is the honest answer to 'is it cheaper': not for a light user with one subscription. The case for local is privacy, offline, control over your own agents, and heavy use where the subscriptions' usage limits start to bite. Our ranking's buyer's guide says the same thing in one line: a $3,000 machine is a bad way to discover you're not one of those people.

What people running this at home are actually saying

We read two weeks of X posts, the month's top threads on r/LocalLLaMA (865,000 members), and the Hacker News threads, pulling only posts we could open and verify. It is a steady undercurrent rather than a wave: typical setup posts get 16 to 70 likes, and the ones that break a thousand are benchmark flexes or dunks. The scale behind it is real, though. Ollama said in July it serves 8.9 million developers a month, a company-provided figure; its repository passed 185,000 GitHub stars; and Hugging Face lists 217,894 models in GGUF format. The month's most-upvoted r/LocalLLaMA thread, at 6,500 points, was Sam Altman saying 'local inference is a terrible allocation of scarce resources', which the community took about as well as you'd expect.

The friction complaints are about defaults, not installs. The most-liked post in our window, 1,266 likes, was about a Linus Tech Tips video:

A reply in that thread put the beginner experience bluntly: 'LM Studio has been a disaster for local AI given how garbage the first time experience and defaults are for normies,' from someone who said they once saw 8 tokens a second, assumed that was all local AI could do, and gave up. That is the GPU-offload and context-default problem from the tools section, seen from the other side.

Apple Silicon as the comfortable path comes up unprompted. 'I've been running open-weight models on my Mac mini and I'm having the time of my life. A Mac Studio is on the way,' wrote one developer on November 1. A Svelte maintainer built a multiplayer game 'using my laptop, Qwen 3.8 27B and Bionic from LM Studio. Fully offline!' At the top end, one developer posted two Mac Studios, an M5 Ultra with 256GB beside an M3 Ultra with 512GB, 'together: 768GB of unified memory', for MLX experiments; 365 likes.

The motive, in the posts' own words: 'No subscription, no usage limit, no API bill,' from a Tamil-language journalist (our translation). On r/LocalLLaMA the sentiment is more pointed: the thread 'Another reason to own your own compute. Your account may get banned in a sneaky update' took 1,700 points, and a user who went from one 3090 to twenty DGX Sparks wrote that from the first 33B Llama 'I knew I wanted that magic-like intelligence locally, mine, so nobody could take it away when I needed it.'

And the realists. On Hacker News's 'What hardware are you using to run local LLMs?' thread this week, a Framework Desktop owner called the Qwen3.8-27B setup in LM Studio 'mostly easier than expected'; a 3090 owner called it 'free intelligence for simple daily tasks'; someone with four clustered Mac Studios said 'don't recommend'; and a 5090 owner reckoned the electricity roughly matched what API credits would cost, 'but no spying'. A thread titled 'Dirt cheap GPUs coming next year' (2,800 points) predicts used-GPU prices will fall when datacenter buildouts are cancelled. Maybe. This week they are double last year's.

The open models you can't run at home, and the cheap way to try them

These are the models the leaderboards talk about, and the honest reason the 'I run the frontier at home' posts are rare is that the frontier is a terabyte. If you want to use them without a datacenter, the hosted route is cheap: every one of them is available by API for cents, and Ollama's cloud tags (`ollama run mistral-large-4:cloud`) use the same commands as your local models. That's a reasonable hybrid: local for private and everyday, cloud for the hard questions, one tool for both.

What we could not verify

• Real-world tokens per second for the current Macs and GPUs on specific models. The bandwidth ceilings in our chart are from the vendors' spec pages; the only measured figures we cite (GLM-4.7-Flash at ~43 tok/s on a 3090, ~121 on a 5090) are community measurements from our ranking, labeled as such.
• The exact share of unified memory macOS reserves from the GPU. Around 25% on 64GB machines is a community figure (the `iogpu.wired_limit_mb` setting); Apple doesn't document it.
• Mac Studio M5 Max prices at 48GB and 128GB. Two outlets reported different 48GB prices on launch day and Apple's configurator doesn't render to our tools; we list only the base prices Apple published.
• Kimi K3.1's open weights, promised for November 4, were not public at publication; K3 is the version in the table.
• LM Studio's current default GPU-offload value. Its CLI docs say the app 'will automatically determine optimal GPU usage' when unspecified; the Developer-mode requirement to see the control is confirmed by the company's own reply.

Questions people ask

Can I run an LLM on a laptop with 8GB?

Yes, small ones: Gemma 4 E4B (6.6GB), Qwen3.5 4B (3.3GB), the DeepSeek-R1 7B distill (4.7GB). Keep conversations short, because the OS wants 2–3GB of that memory too. LM Studio says 8GB 'may work' and recommends 16GB; so do Microsoft's Copilot+ PC specs. If you're buying anything, buy 16GB minimum and 32GB if you can.

Mac or PC for local AI?

For most people, a Mac: unified memory means no offload settings, Apple writes the engine (MLX), the apps support it first, the boxes are silent and sip power, and Apple held prices this generation while GPUs doubled. A PC wins if you already own a 24GB+ card, if you need CUDA for anything beyond inference (fine-tuning, image and video models), or if you want the absolute fastest consumer chip and can stomach $6,600 for an RTX 5090.

Is it really private?

The inference is. Both Ollama and LM Studio state in their docs that chat content never leaves the device, and you can confirm it with the Wi-Fi off. The caveats: the apps check for updates and search Hugging Face for models unless you stop them; Ollama's optional cloud models run on Ollama's servers (disable with `OLLAMA_NO_CLOUD=1`); and if you expose the server to the internet, your privacy problem is other people, not the vendor.

How hard is it, honestly?

Install: trivially easy on every platform now. Getting a good result: easy if you pick a model that fits your memory, which is why this guide spends so long on that one number. The three things that produce 'it's slow' or 'it's broken' posts are a model that spilled out of VRAM, a GPU that isn't being used because a setting is hidden (LM Studio's Developer mode), and a context window set to maximum on a machine that can't hold the cache. Check those three and nearly every problem goes away.

Will a local model replace my ChatGPT or Claude subscription?

For drafting, summarising, private documents, and coding help, a 27B-class model does most of the job. For hard reasoning and long autonomous tasks, no: the gap between 20 and 58 on the same intelligence index is visible in daily use. Most people we read end up hybrid, and the cost curve above shows why that's rational: one subscription stays cheaper than a machine for four years.