Lesson 26 of 36 · Track 3: Build with AI

How to run an LLM locally on your laptop (Ollama and LM Studio)

Beginner13 min6 steps1 videoFree · no login
The short answer

Open-weight models like Gemma, gpt-oss, Llama, Qwen, DeepSeek and Mistral can be downloaded and run on your own computer, so your chats stay on your machine and work without internet once the model is downloaded. The two easiest tools are Ollama (one install command, then "ollama run") and LM Studio (a desktop app with a chat window). The main limit is memory: aim for 16 GB of RAM, start with a small model, and use a 4-bit quantized version.

  1. 1

    Open-weight vs closed models: what is the difference?

    A model’s weights are the billions of numbers it learned in training. With a closed model, like the ones behind ChatGPT, Claude or Gemini, the weights stay on the company’s servers; you use the model through an app or API. With an open-weight model, the company publishes the weights so anyone can download and run them.

    Licences differ. OpenAI’s gpt-oss models use the permissive Apache 2.0 licence. Google DeepMind calls Gemma "a family of open-weights" models. Meta’s Llama comes with its own licence file, which Meta says covers researchers and commercial use. "Open-weight" is not always the same as fully open source, so read the licence before building a product.

    Key idea

    Open-weight means you can download the model and run it yourself, under the terms of its licence.

  2. 2

    Why run a model on your own computer?

    Privacy is the big one. Ollama’s FAQ says it runs locally and "conversation data does not leave your machine". That matters for a CV with your phone number, company documents or medical notes.

    Other reasons: it works offline once the model is downloaded, there is no per-message fee, and you learn how models really behave. The trade-off is honest: a model that fits on a laptop is smaller and usually less capable than the top cloud models, and it can be slow on older hardware.

    • Good for: private notes, summarising your own files, offline use, learning and tinkering.
    • Not ideal for: hard reasoning, long research, or anything needing the newest facts.
  3. 3

    How much RAM do you need? Quantization explained

    A model must fit in memory to run. RAM is your computer’s main memory; VRAM is the separate memory on a graphics card (GPU), which runs models much faster. Apple Silicon Macs share one pool, called unified memory. LM Studio’s system requirements (as of Nov 2026) advise 16 GB of RAM or more; 8 GB Macs can work with small models, and on Windows at least 4 GB of dedicated VRAM is suggested.

    Quantization makes models smaller by storing each number with fewer bits, like saving a photo as a smaller JPEG. LM Studio’s docs describe quantized files as copies of the same model at different fidelity: smaller files, slightly lower quality. Its advice is to pick a 4-bit option or higher if your machine can run it.

    Rough size of an 8-billion-parameter model (weights only)
    16-bit:  8 billion x 2 bytes   = about 16 GB
8-bit:   8 billion x 1 byte    = about 8 GB
4-bit:   8 billion x 0.5 bytes = about 4 GB
(Leave extra room for your apps and the conversation.)
    Key idea

    Parameters times bits decides the size. A 4-bit version is usually the sweet spot for laptops.

  4. 4

    Your first run with Ollama

    Ollama is a free, open-source tool that downloads and runs models from the command line, with a desktop app on Mac and Windows. Install it with the official command below, or download the app from ollama.com. Then run a small model; Ollama’s quickstart uses Gemma 4’s e2b version, about a 7.2 GB download, and recommends 8 GB of available VRAM or Mac unified memory.

    Type your message and press Enter. Type /bye to quit. The command "ollama ps" shows whether the model is running on your GPU or CPU. Note that Ollama also offers cloud models when you sign in; those run on Ollama’s servers, not your laptop, so pick a local model when privacy matters.

    Install and run (commands from Ollama’s official README and quickstart)
    # macOS or Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows (PowerShell)
irm https://ollama.com/install.ps1 | iex

# Chat with a small model
ollama run gemma4:e2b

# See what is loaded, and where
ollama ps
  5. 5

    Prefer buttons? Your first run with LM Studio

    LM Studio is a desktop app with a chat window, so you never touch a terminal. As of Nov 2026 it runs on Apple Silicon Macs (not Intel Macs), Windows x64 and ARM, and Linux.

    Its built-in downloader pulls models from Hugging Face, the largest public hub of open models; you can search by name or paste a Hugging Face link.

    1. Install LM Studio
    2. Discover tab: search a model
    3. Pick a 4-bit (Q4) file that fits
    4. Chat tab: load the model
    5. Chat offline
  6. 6

    Which open models should you try in 2026?

    New open models arrive every month, so treat this as a starting list as of Nov 2026. Start small, check it runs smoothly, then move up.

    • Gemma 4 (Google DeepMind): open-weights family; the small e2b version is Ollama’s quickstart model.
    • gpt-oss (OpenAI, August 2025, Apache 2.0): gpt-oss-20b runs within 16 GB of memory; gpt-oss-120b needs a single 80 GB GPU.
    • Llama (Meta): Llama 3.2 has small 1B and 3B models; Llama 4 Scout and Maverick (April 2025) are much larger.
    • Qwen (Alibaba), DeepSeek and Mistral: popular families with many sizes, all listed in Ollama’s library.
    Try OpenAI’s open model if you have 16 GB or more
    ollama run gpt-oss:20b
From the people who built it

Straight from the source

The original idea, then the same idea taken apart, then how you use it today.

OpenAI
Maker of ChatGPT, which also publishes the open-weight gpt-oss models
gpt-oss (model release and GitHub repository, August 2025)
1 · Understand
Two open-weight reasoning models under the Apache 2.0 licence: gpt-oss-120b for heavy work on one 80 GB GPU, and gpt-oss-20b for local use within 16 GB of memory.
2 · Break it down
  • Mixture-of-experts: gpt-oss-20b has 21B parameters but only 3.6B are active for each token, which keeps it fast.
  • The weights ship already quantized (MXFP4), which is why 20B fits in 16 GB.
  • Apache 2.0 lets you use and modify it commercially without copyleft rules.
3 · Use it
If your laptop has 16 GB or more, run "ollama run gpt-oss:20b" and compare its answers with a cloud chatbot on the same question.
Meta
Maker of the Llama family of open-weight models
Llama models repository (Llama 3.1 to Llama 4)
1 · Understand
Meta publishes Llama weights in many sizes, from 1B and 3B models (Llama 3.2) that fit small machines to Llama 4 Scout and Maverick, under its own licence for researchers and commercial use.
2 · Break it down
  • Sizes matter: the 1B and 3B models are small enough for ordinary laptops.
  • Bigger versions need far more memory but answer better.
  • The licence is Meta’s own, so check it before using Llama in a product.
3 · Use it
Pick the smallest size that answers your kind of question well, and only move up when it clearly falls short.
6 terms

Words to know

Open-weight model
A model whose trained weights are published, so you can download and run it yourself under its licence.
Parameters
The numbers inside a model learned during training; more parameters usually means more capable but larger.
RAM / VRAM
RAM is your computer’s main memory; VRAM is the faster memory on a graphics card. The model has to fit in them.
Quantization
Storing a model’s numbers with fewer bits (for example 4-bit) to make it smaller and faster, with a small loss in quality.
Ollama
A free, open-source tool to download and run open models from the command line or its app.
Hugging Face
The largest public website for sharing open models and datasets; LM Studio downloads from it.
1 video · plays here

Watch

Hand-picked from the people who explain this best. The video loads only when you press play.

What is Ollama? Running Local LLMs Made Simple
IBM Technology · 7 min

Cedric Clyburn shows how Ollama runs models on your own machine and why that helps privacy and cost. Watch before step 4.

Hands-on · 10 minutes or less

Try it

Chat with an AI that never leaves your laptop

  1. 1Check your memory: on Windows, Task Manager, then Performance; on a Mac, Apple menu, then About This Mac.
  2. 2Install Ollama from ollama.com (or LM Studio from lmstudio.ai if you prefer an app).
  3. 3Run "ollama run gemma4:e2b" and wait for the download to finish (about 7.2 GB).
  4. 4Turn off Wi-Fi and ask: "Explain quantization to a 12-year-old in 3 sentences." It still answers, because the model is on your computer.
  5. 5Run "ollama ps" in a second terminal and note whether it says GPU or CPU. If replies are very slow, try a smaller model or close other apps.
Ollama on GitHub (install and quickstart)
3 questions · instant feedback

Check yourself

Pick an answer to see if you are right, and why.

Question 1

Question 1 of 3

What does "open-weight" mean?

Question 2

Question 2 of 3

Roughly how much memory do the weights of an 8-billion-parameter model need at 4-bit?

Question 3

Question 3 of 3

You want your chats to stay completely on your machine. In Ollama, what should you avoid?

Lesson 26 of 36

Done? Keep going

Mark this lesson complete to keep your place. Progress is saved on this device, no login.

Up next · Lesson 27 · 14 minYour first AI automation in n8nn8n is a workflow automation tool: you join apps together as boxes on a canvas, and the workflow runs by itself when something happens. You can add an AI step that reads, summarises or sorts text with a model like GPT, Claude or Gemini. In this lesson you build one real workflow: a form is submitted, an AI summarises it, and the summary lands in your email or Slack.← Previous: How voice AI agents work
People also ask

Questions people ask

Can I run an LLM on a laptop with 8 GB of RAM?

Yes, with small models. LM Studio says 8 GB Macs can work if you stick to smaller models and modest context sizes, and Ollama’s quickstart model gemma4:e2b recommends about 8 GB of VRAM or Mac unified memory. 16 GB gives you far more choice.

Is Ollama free and safe?

Ollama is free and open source. Its FAQ says local conversation data does not leave your machine. It needs internet to download models, and its optional cloud models run on Ollama’s servers, so choose local models for private work.

Ollama or LM Studio: which should a beginner use?

If you are comfortable typing one command, Ollama is quick and also works as a local API for your own apps. If you want a normal app with buttons and a chat window, start with LM Studio. Both run the same kinds of open models.

11 sources · checked Nov 2026

Sources and further reading

Related on this site
Previous: How voice AI agents workNext: Your first AI automation in n8nChatGPT vs Claude vs GeminiTokens and context windowsPrompting vs RAG vs fine-tuningAI engineer career guideFree AI skills resources
Free · No signup to start

Going for an AI role? Practise the interview.

Talk to an AI interviewer about what you just learned and get feedback on every answer.

Start a free mock interview