Ollama on 8GB RAM: which models work in 2026

Have a laptop with 8GB of RAM and want to run AI locally? This article is a breakdown: what works, what barely runs, and what's not even worth downloading. No illusions, with specific models and commands for each task. If you're not yet familiar with Ollama — start with our introductory article on what Ollama is and why developers are massively switching to local AI.

Updated: October 2026. I've checked model sizes against the Ollama library and removed recommendations that don't fit within 8GB.

📚 Table of Contents

• 📌 RAM and Available Memory
• 📌 Code and Copilot Replacement
• 📌 Text and Chat Models
• 📌 Reasoning and Logic
• 📌 CPU vs GPU vs Apple Silicon
• 📌 Quantization (Q4 vs Q8)
• 📌 Ollama Config for 8GB
• 📌 What Doesn't Work on 8GB
• 📌 Real Speed Tests
• ❓ FAQ
• ✅ Conclusions

🎯 How Much RAM is Actually Left for the Model

Short answer: With 8GB of RAM, 3–5GB is realistically available for an AI model. The rest is taken by the operating system, browser, and background processes. This dictates the main rule: models weighing up to 3–3.5GB on disk (usually 3–4B parameters in 4-bit quantization) work comfortably on 8GB, while 7–8B models operate on the edge.

8GB RAM is not 8GB for the model. It's 8GB minus the OS, minus Chrome, minus everything you forgot to close.

Before choosing a model, you need to understand your real memory budget. Here's a typical breakdown on a system with 8GB RAM:

• ✔️ Operating System: 1.5–2.5GB (macOS closer to 2.5, Windows — 2, Linux — 1.5)
• ✔️ Browser (5–10 tabs): 1–2GB
• ✔️ IDE (VS Code / IntelliJ): 0.5–1.5GB
• ✔️ Background Processes: 0.3–0.5GB

Remaining for the model: 3–5GB.

According to LocalLLM.in, a 7B parameter model in Q4_K_M quantization takes approximately 4–5GB, plus 1–2GB for KV cache and system overhead. This means: a 7B model on 8GB is possible, but on the edge, and it's best to close everything unnecessary.

📏 Disk Size ≈ Memory Size

The model size in the Ollama library is almost equal to the memory volume its weights will occupy. This is in addition to the KV cache (which grows with the context window) and service memory. Therefore, don't look at the number of parameters in the name, but at the actual tag size. Here's what the Ollama library shows for the models we're discussing in the article:

🔗 Sources: model pages in the Ollama library, size summary on NeedToKnowIT, Gemma 4 sizes on iTechGuides. Ollama tags and sizes are updated, so check the model page before downloading.

Rule of thumb for 8GB:

• ✔️ Comfort zone: models up to ~3.5GB on disk (1–4B parameters in Q4_K_M) — leaves space for IDE and browser
• ✔️ Working zone: 5–5.5GB models (7–8B in Q4_K_M) — requires closing everything unnecessary
• ❌ Red zone: anything weighing over ~6GB, and 13B+ models — freezes or swaps to disk

⚠️ Mac Specifics: on Apple Silicon, memory is unified, but macOS by default allocates only a portion to the GPU (on Macs with 8GB, this is approximately 5–5.5GB). If the model is larger, Ollama moves some layers to the CPU, and the speed drops sharply.

Conclusion: Before choosing a model, close your browser, check ollama ps, and see the actual remaining memory. On 8GB, every gigabyte is worth its weight in gold.

🎯 For Code: Which Model Will Replace Copilot on 8GB

Code Autocompletion For code autocompletion on 8GB, the best options in 2026 are Qwen3.5 4B, Qwen3 4B, or Phi-4 Mini (3.8B) in Q4_K_M quantization. Qwen3.5 4B is the newest of the three, with multimodality and a large context, but also the heaviest (3.4GB). Qwen3 4B and Phi-4 Mini weigh 2.5GB each.

GitHub Copilot costs $10/month. A local model for code is $0/month and works offline. The only question is which model will run on your hardware.

Coding is a task where even small models can be useful. Autocompletion, function generation, code explanation, writing tests — you don't need GPT-4 for this, you need a fast and accurate model that understands syntax.

Top Models for Code on 8GB

1. Qwen3.5 4B (Q4_K_M) — ~3.4GB

Released in March 2026 as part of the small Qwen3.5 series (0.8B, 2B, 4B, 9B). The model is natively multimodal (text, images), supports thinking mode and tool calling, and its 256K token context window covers most real-world codebases. The Apache 2.0 license allows for free commercial use. This is a versatile model, not a narrowly specialized coder, so for pure code consider the next option as well.

ollama pull qwen3.5:4b
ollama run qwen3.5:4b "Write a Python array sorting function"

⚠️ About thinking mode: in reasoning mode, the model first generates "thoughts," making the response longer and slower. For autocompletion and simple queries, it's better to disable it. In Ollama, thinking is controlled by the --think flag or the /set nothink command in chat. For Qwen3 models, text switches /think and /no_think in the prompt also work.

2. Qwen3 4B (Q4_K_M) — ~2.5GB

Released in April 2025, it remains a reliable choice: lighter than Qwen3.5 4B by ~0.9GB, which is noticeable on 8GB. According to the Ollama Library, its response quality approaches that of much larger previous-generation models. If multimodality isn't needed and memory is scarce, this is the optimal balance.

ollama pull qwen3:4b
ollama run qwen3:4b "Find the error in this Java code: ..."

3. Phi-4 Mini (3.8B) — ~2.5GB

A compact model from Microsoft with a 128K context. According to SitePoint, it runs comfortably on systems with 8GB, including MacBook Air with M1. It handles autocompletion, simple explanations, and light chat tasks well.

ollama pull phi4-mini
ollama run phi4-mini "Explain the difference between HashMap and TreeMap in Java"

4. Qwen2.5-Coder 1.5B or 3B — for background autocompletion

If you need the model to run in the background alongside VS Code, browser, and terminal, choose the lightest specialized coding models. The Qwen2.5-Coder series has 1.5B (~1GB) and 3B (~2GB) versions and is designed specifically for inline autocompletion. It's a more modern replacement for the old deepseek-coder:1.3b (from 2023), which I recommended in the previous version.

ollama pull qwen2.5-coder:3b
# even lighter option
ollama pull qwen2.5-coder:1.5b

What to choose?

• Qwen2.5-Coder 1.5B / 3B — background autocompletion, working with an open browser
• Qwen3 4B — function generation and code explanation with minimal size
• Qwen3.5 4B — the same plus UI screenshot analysis and long contexts
• Phi-4 Mini — a versatile model for code and text tasks

Conclusion: You can code with local AI on 8GB. Don't expect GPT-4 quality — but for daily autocompletion, boilerplate generation, and code explanations, it's more than enough. However, Qwen3.5 4B turned out to be heavier than I previously stated after checking its size (3.4GB instead of 2.5GB), so on 8GB, the choice between it and Qwen3 4B depends on how much memory you're willing to allocate.

🎯 For Text and Communication: Chat, Translation

For Text Tasks For text tasks on 8GB, the optimal choices are Llama 3.2 3B for general chat, Gemma 3 4B for multimodality and multilingualism, or Phi-4 Mini if you need analytics and CPU-only operation. All three leave space for other software and work stably without hardware upgrades.

Not every task requires GPT-4. Summarizing text, answering questions, retelling an article — a model weighing less than one 4K movie can handle this.

Text tasks are the broadest category: from simple chat to document analysis and translation. On 8GB, there's plenty to choose from. If you've already downloaded qwen3:4b or qwen3.5:4b from the previous section — they handle text tasks excellently as well. But if you're looking for a specialized recommendation specifically for text, here's an up-to-date list.

🇺🇦 If you write in Ukrainian. Not all small models work equally well with Ukrainian. For example, Llama 3.2 officially supports 8 languages (English, German, French, Italian, Portuguese, Hindi, Spanish, Thai), and Ukrainian is not among them. Qwen3 and Gemma claim support for 100+ languages, including Ukrainian. Before choosing a model for daily work, give it a few of your typical texts and compare the results.

Top Models for Text on 8GB

1. Llama 3.2 3B (Q4_K_M) — ~2GB

According to StudyHUB, Llama 3.1/3.2 is the most popular model on Ollama with over 111 million downloads. The 3B version is a lighter variant but retains quality in general conversations, summarization, and question answering. It's the lightest model on our list. For comparison: Llama 4 Scout, released in 2025, weighs around 67GB in Ollama (an MoE model with 17B active parameters), so it's not suitable for 8GB. Within the Llama family, 3.2 3B remains the best choice for 8GB.

ollama pull llama3.2:3b
ollama run llama3.2:3b "Retell the main idea of this text: ..."

2. Gemma 3 4B (Q4_K_M) — ~3.3GB

A model from Google DeepMind that accepts text and images and claims support for over 140 languages. It fits within 8GB, leaving space for an IDE and browser. This is a good choice if you need multimodality and multilingualism, including Ukrainian.

ollama pull gemma3:4b
ollama run gemma3:4b "Write a short description for this product: ..."

⚠️ What about Gemma 4? The new Gemma 4 generation (April 2026) is multimodal, with thinking mode and 128K context, but the standard tags in Ollama are significantly heavier: gemma4:e2b is 7.2GB, and gemma4:e4b (the default tag) is 9.6GB. They do not fit on 8GB. There are quantized QAT versions that are reported to weigh around 4.3GB (e2b), but check the current tags on the Gemma 4 page before downloading. More details about Gemma 4 are in the article Gemma 4: Full Overview — Sizes, License, Comparison with Gemma 3 and about thinking mode — Reasoning Mode in Gemma 4: How to Enable, When Needed, and What It Costs.

3. Phi-4 Mini (3.8B) — ~2.5GB

One of the few 3–4B models with a 128K token context window, allowing it to analyze long documents. According to PromptQuorum, it works well on CPU without discrete graphics, making it suitable for laptops without a GPU and even Raspberry Pi 4/5. Check Ukrainian quality with your own examples.

ollama pull phi4-mini
ollama run phi4-mini "Translate to Ukrainian: The quick brown fox jumps over the lazy dog"

What to choose?

• ✔️ Lightest option for general chat → Llama 3.2 3B
• ✔️ Multimodality (text + images) and multilingualism → Gemma 3 4B
• ✔️ Analytics, long documents, and CPU-only → Phi-4 Mini
• ✔️ One tool for code and text → Qwen3 4B or Qwen3.5 4B (see previous section)

Conclusion: For text tasks, 8GB is a comfortable territory. 2–4B models work quickly, leave space for other programs, and provide quality sufficient for most daily needs. The main rule: choose based on the tag size in Ollama, not on the fancy name of a new generation.

🎯 For Reasoning, Logic, Code Debugging

For tasks requiring step-by-step thinking — mathematics, logic puzzles, debugging complex code — there are three realistic options on 8GB: DeepSeek R1 8B as a classic "thinking" model, Qwen3 8B for multilingual reasoning, and Phi-4 Mini Reasoning as a lighter option with chain-of-thought.

A regular model answers immediately. A reasoning model first thinks — step by step — and then answers. Like the difference between "guessing an answer" and "calculating on paper."

Reasoning models are a relatively new category. They work on the chain-of-thought principle: they break down a task into steps, check intermediate results, and only then form the final answer. Reasoning mode is now available not only in heavy 8B models but also in compact 3–4B variants.

What Works on 8GB

1. DeepSeek R1 8B (Q4_K_M) — ~5.2GB

According to StudyHUB, on tasks involving math, logic puzzles, and technical reasoning, this model yields better results than Llama 3.1 of the same size. Important: this is not the original DeepSeek R1, but a distilled version — the reasoning skills of a large model have been transferred to a smaller one. Before the final answer, it generates visible reasoning steps within <think> tags — useful for debugging to understand why the model reached a particular conclusion. The trade-off: it responds slower and requires almost all available memory on an 8GB system.

ollama pull deepseek-r1:8b
ollama run deepseek-r1:8b "Find the error in this SQL query: SELECT * FROM users WHERE id = '5' AND active = true GROUP HAVING count > 1"

⚠️ Important: DeepSeek R1 8B takes ~5.2GB on disk, so on an 8GB system, it's on the edge — you need to close your browser, IDE, and everything else. On macOS with unified memory, it runs more stably than on Windows with integrated graphics.

2. Qwen3 8B (Q4_K_M) — ~5.2GB

According to LocalLLM.in, Qwen3 8B is a strong alternative for reasoning tasks, especially in mathematics and multilingual scenarios. It supports thinking mode: it can be toggled on and off without reloading the model (/think and /no_think in the prompt). Do not confuse it with qwen3.5:9b: this model weighs 6.6GB and does not fit on 8GB; it's for systems with 16GB.

ollama pull qwen3:8b
ollama run qwen3:8b "Solve: if 3x + 7 = 22, what is x?"

3. Phi-4 Mini Reasoning (3.8B) — ~3GB

A specialized reasoning variant from Microsoft, designed for multi-step mathematical problem-solving in memory-constrained environments: symbolic computation, formal proofs, complex textual conditions. According to Morph, it's one of the few reasoning models that leaves space for parallel operation of an IDE and browser. Unlike DeepSeek R1 8B, it occupies almost half the memory and keeps the system responsive. On complex tasks, it's inferior to 8B models, but for daily debugging and analytics, it's sufficient.

ollama pull phi4-mini-reasoning
ollama run phi4-mini-reasoning "Find the algorithm's complexity and explain step-by-step: ..."

What to choose?

• ✔️ Code debugging and logic puzzles → DeepSeek R1 8B
• ✔️ Mathematics and multilingual reasoning → Qwen3 8B
• ✔️ Reasoning with an open IDE and browser → Phi-4 Mini Reasoning (~3GB)

Conclusion: Reasoning on 8GB is no longer just "on the edge of comfort." Phi-4 Mini Reasoning allows for step-by-step thinking at ~3GB — without needing to close everything else. For more complex tasks, DeepSeek R1 8B and Qwen3 8B remain the standard, but they require almost all available memory. If you plan to regularly work with heavy reasoning tasks — an upgrade to 16GB will open access to the 14B class and Qwen3.5 9B, where the quality difference is already significant.

🎯 CPU vs GPU vs Apple Silicon — where 8GB is not 8GB

8GB on a Mac M1 and 8GB on a Windows laptop with Intel is a completely different experience. Apple Silicon uses unified memory, where all memory is accessible to both the CPU and GPU simultaneously. On a standard PC, RAM and VRAM are separate pools, and this is critical for AI models.

A Mac with 8GB is the best option among laptops with this amount of memory. A Windows laptop with 8GB and Intel HD Graphics is a struggle for every megabyte.

Apple Silicon (M1/M2/M3/M4) — the best scenario for 8GB

On Apple Silicon, all RAM is unified memory. This means the GPU part of the chip has access to the same 8GB as the CPU. Ollama automatically uses Metal for acceleration — without any additional settings.

Approximately, 3-4B models on M1 with 8GB deliver 15-25 tokens/sec, and 7-8B models — 8-12 tokens/sec, and that's only if the model fits entirely into memory (see section 1 about GPU limits on Mac). According to SitePoint, Phi-4 Mini on an M1 MacBook Air runs at approximately 15-20 tok/s, which is sufficient for daily work.

💡 Measure it yourself. Run the model with the --verbose flag: ollama run phi4-mini --verbose. After each response, Ollama will show the eval rate — the actual generation speed on your hardware. This is more accurate than any tables from the internet.

⚠️ Note on MLX: at the end of March 2026, Ollama 0.19 received Apple's MLX backend (still in preview), which provides almost a 2x speed increase on large models. However, it is designed for Macs with 32GB+ unified memory: according to Ollama's blog and the Switching to Mac guide, Macs with 8GB and 16GB remain on the previous llama.cpp + Metal without speed changes. Details of this feature change from release to release, so check the release notes for the latest Ollama version. If you plan to upgrade to a Mac with 32GB+, MLX will be a significant bonus.

Windows / Linux with discrete GPU (RTX 3060, RTX 4060) — a good scenario

If you have a discrete graphics card with 6-8GB of VRAM, the model will be fully loaded into GPU memory, and system RAM will remain for the OS and software. According to LocalLLM.in, on an RTX 4060 (8GB VRAM), a 7B model delivers 40+ tokens/sec — the fastest option overall. The main condition is that the model must fit into VRAM. If it doesn't fit, some layers will go to the CPU, and the speed will drop sharply. Check with the command ollama ps: the Processor column should show 100% GPU.

Windows / Linux without GPU (Intel HD / AMD Radeon iGPU) — a difficult scenario

Without a discrete GPU, the model runs entirely on the CPU. Ollama will still launch — but the speed drops to 3-6 tokens/sec for 7B models. For lighter 3-4B models (Phi-4 Mini, Llama 3.2 3B), the actual speed on a modern CPU is approximately 8-12 tok/s — quite acceptable for daily tasks. According to LocalLLM.in, CPU-only inference is acceptable for batch tasks but frustrating for interactive use with large models.

Additionally, system RAM is shared between the OS, software, and the model — on 8GB this is very tight.

Summary table

Values are approximate, gathered from third-party tests. Actual speed depends on the chip, context, and background processes.

Conclusion: If you have a Mac M1+ with 8GB, you are in the best position for local AI on a budget hardware, and 3-4B models will work quite comfortably. Ollama's MLX backend is currently aimed at Macs with 32GB+, so for 8GB, the speed remains as it is. If you have Windows without a GPU, focus on 3-4B models: they keep the system responsive. More details on installation on different OSs are in the article How to Install Ollama on Mac, Windows, and Linux.

🎯 Quantization in simple terms: Q4 vs Q8 and what to choose for weak hardware

Short answer: Quantization is model compression that reduces its size by 2-4 times with minimal loss of quality. For 8GB, the optimal choice is Q4_K_M: the best balance between size, speed, and response quality.

Quantization is like JPEG for photos. The file is smaller, the difference is almost imperceptible. But if you compress too much, the quality will noticeably decrease.

When you see tags like :7b-q4_0, :8b-instruct-q8_0, or :3b-q4_k_m in an Ollama model name — this indicates the quantization level. The number after "q" is the number of bits per parameter. For most models, the default tag in Ollama is already Q4_K_M, so you don't need to specify anything extra.

Quantization levels: what the tags mean

• ✔️ Q8 (8-bit): maximum quality, largest size. For a 7B model — ~8GB. Won't fit in 8GB RAM.
• ✔️ Q5_K_M (5-bit): an intermediate option between Q4 and Q8. For 7B — ~5.5GB. Only suitable for 8GB RAM if you have a GPU with 6-8GB VRAM and require higher precision.
• ✔️ Q4_K_M (4-bit, K-quant medium): optimal balance. For 7B — ~4-5GB. Recommended for 8GB systems.
• ✔️ Q4_K_S (4-bit, K-quant small): slightly smaller than Q4_K_M, slightly lower quality.
• ✔️ IQ4_XS (importance matrix, 4-bit): a newer format. According to RunAIHome, it provides almost the same quality as Q4_K_M but takes ~400MB less for an 8B model. Useful when Q4_K_M barely fits. Available as a tag on Hugging Face, not always present in the Ollama Library.
• ✔️ QAT versions: some models (e.g., in the Gemma family) have variants trained with quantization in mind. If there's a tag marked qat for the model you need, compare it with Q4_K_M in terms of size and quality.
• ⚠️ Q2_K (2-bit): minimum size (~2.5GB for 7B), but noticeable quality degradation. An extreme option.

The "K" suffix denotes newer quantization methods (K-quant) that more intelligently distribute precision across model layers. K-quant tags are always better than legacy variants (q4_0, q4_1) at the same size.

How much do models of different quantizations weigh

Q4_K_M size is the actual size of the default tags from the Ollama library, Q8 and Q2_K are approximate estimates.

Sources: Ollama library, LocalAIMaster, RunAIHome. Note: for some new models, the default tag is not Q4_K_M but a heavier one (like Gemma 4, see section 3), so always check the actual tag size.

Rule for 8GB: always choose Q4_K_M. If it doesn't fit, reduce the model size (3B instead of 7B), not the quantization level (Q2 instead of Q4). A smaller model with Q4 will provide better quality than a larger one with Q2. Exception: if Q4_K_M literally doesn't fit by 100-400MB, try IQ4_XS if such a tag is available for the model on Hugging Face.

More on compression techniques and their impact on quality — in our article on model quantization (INT4, INT8).

Conclusion: Q4_K_M is the gold standard for 8GB and has not lost its relevance. Don't give in to the temptation to download Q8 "for quality" — the model won't fit into memory, and you'll get disk swapping. Alternatives on the horizon are IQ4_XS and QAT versions: slightly smaller size or better quality, but not for every model.

🎯 Ollama settings for maximum performance on weak hardware

A few environment variables and one habit (closing unnecessary things) are all it takes to squeeze the most out of 8GB. The setup takes a few minutes, and the difference in stability is noticeable.

On powerful hardware, Ollama "just works." On weak hardware, you need to help it not waste memory on what you don't need.

By default, Ollama uses a context window of 4096 tokens, can hold up to three models in memory simultaneously, and processes one parallel request per model. On 8GB, holding multiple models is an unnecessary luxury. Here's what's worth configuring:

🔗 Sources: Ollama documentation (FAQ), DEV Community, SSD Nodes.

Basic environment variables

# Keep only one model in memory
export OLLAMA_MAX_LOADED_MODELS=1

# One parallel request (this is the default value, setting for reliability)
export OLLAMA_NUM_PARALLEL=1

# Context window: 2048 for simple tasks, 4096 for reasoning
export OLLAMA_CONTEXT_LENGTH=2048

⚠️ Attention: in the previous version of the article, I specified the variable OLLAMA_CTX_SIZE. There is no such variable in the Ollama documentation; the correct name is OLLAMA_CONTEXT_LENGTH. If you installed the old one, the setting would not work.

Variables for saving KV cache (GPU and Apple Silicon)

If you are running Ollama on a GPU (NVIDIA, AMD) or Apple Silicon (M1+) — set these two variables additionally. They have existed since late 2024 and reduce the KV cache by approximately half without noticeable quality loss for most tasks.

# Flash Attention — a prerequisite for KV cache quantization
export OLLAMA_FLASH_ATTENTION=1

# 8-bit KV cache — approximately half the memory for cache
export OLLAMA_KV_CACHE_TYPE=q8_0

⚠️ Important: OLLAMA_KV_CACHE_TYPE only works if OLLAMA_FLASH_ATTENTION=1 is enabled. According to SSD Nodes, q8_0 saves approximately half the memory compared to f16, and q4_0 saves approximately a quarter, but q4_0 can noticeably degrade quality, so q8_0 is sufficient for 8GB. Cache quantization may slightly reduce generation speed. If after enabling it a model doesn't load, suspect OLLAMA_KV_CACHE_TYPE first and try removing it. These variables will not be useful for CPU-only systems.

Where to set variables

• 🍎 macOS (Ollama app): via launchctl setenv NAME value, then restart Ollama. Lines in ~/.zshrc will only work if you manually start the server with the command ollama serve from the terminal
• 🐧 Linux (systemd): in systemctl edit ollama.service add lines Environment="NAME=value" in the [Service] section and restart the service
• 🪟 Windows: set user or system environment variables and restart Ollama from the system tray

The context can also be changed without environment variables: in chat with the command /set parameter num_ctx 4096 or with the num_ctx parameter in Modelfile.

Before running a model

It sounds trivial, but on 8GB it's critical:

• ✔️ Close your browser or leave a maximum of 2-3 tabs open
• ✔️ Close Slack, Discord, Spotify — each program consumes 200-500MB
• ✔️ Check current usage: ollama ps will show loaded models and GPU/CPU utilization
• ✔️ If an old model is still in memory — ollama stop model_name

Modelfile for fine-tuning

If you want more control, create a Modelfile with optimized parameters:

FROM phi4-mini
PARAMETER num_ctx 2048
PARAMETER num_thread 4
PARAMETER temperature 0.7

Save the file as Modelfile and create your own model version:

ollama create phi4-mini-lite -f Modelfile
ollama run phi4-mini-lite

num_ctx 2048 — reduces the context window (less RAM for KV cache). num_thread 4 — limits the number of CPU threads to keep the system responsive. This is a Modelfile parameter, not an environment variable.

A step-by-step guide to installation and first run is in the article How to Install Ollama on Mac, Windows, and Linux: Full Guide 2026.

Conclusion: One model in memory + sensible context + closed unnecessary programs = stable operation on 8GB. If you have a GPU or Apple Silicon, add OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0: according to Ollama's documentation, this reduces KV cache memory by approximately half. Without basic settings, even a light model can cause disk swapping.

🎯 What NOT to try on 8GB

Short answer: Models 13B+, any models in Q8 quantization, attempts to hold two models simultaneously, and MoE models with a large total size — guaranteed disappointment on 8GB. Here are the typical pitfalls that almost everyone encounters when trying local AI on such hardware.

Everyone who has worked with Ollama on 8GB goes through the same stage: "Maybe 14B will fit after all?" No, it won't.

❌ 13–14B models and larger

Qwen3 14B (9.3GB in Ollama library), DeepSeek R1 14B (9GB), CodeLlama 13B — even in Q4 quantization, they require 8-9GB just for the model weights. Add KV cache and the OS, and you'll get a system that constantly swaps to disk: loading takes minutes, and speed drops to 1-2 tokens per second with constant pauses. This is unworkable for interactive use.

❌ Any 7B model in Q8 quantization

The Q8 version of a 7B model weighs about 8GB — that's all your RAM. The OS doesn't magically disappear, so the system freezes or starts swapping. Always use Q4_K_M for 7B models on 8GB.

❌ Two models simultaneously

By default, Ollama can hold multiple models in memory (up to three). On 16GB, this is convenient — you can switch between models instantly. On 8GB, this is a recipe for a swap storm. Keep OLLAMA_MAX_LOADED_MODELS=1 and don't forget to ollama stop before loading another model.

❌ Large context windows (8K+ tokens)

Every doubling of the context window requires hundreds of megabytes more for KV cache. On 8GB, keep the context at a maximum of 4096 tokens (this is the default value anyway). You won't be able to pass a 10-page document to the model entirely; it needs to be split into parts. Partially helpful are OLLAMA_FLASH_ATTENTION=1 + OLLAMA_KV_CACHE_TYPE=q8_0 from the settings section — they reduce the KV cache by approximately half. But 8K+ on 8GB remains a risk zone.

❌ MoE models with large total size (Mixtral, Llama 4 Scout, Qwen3.5/3.6 35B-A3B)

MoE (Mixture of Experts) architecture is misleading with its names. Mixtral 8x7B activates only 2 out of 8 "experts" per token — but all 8 must be in memory simultaneously, and that's 26+ GB in Q4. Llama 4 Scout (2025) looks like "17B," but in the Ollama library, its llama4:16x17b tag weighs about 67GB. Models like Qwen 35B-A3B activate only 3B parameters per token, but keep all 35B in memory — that's 20+ GB in Q4. The rule is simple: look at the total tag size in the Ollama Library, not the number of active parameters.

❌ Thinking mode with a tiny context

Qwen3 8B, Qwen3.5 4B, and Phi-4 Mini Reasoning generate "thinking tokens" before the final answer — sometimes thousands of internal reasoning tokens. These tokens also take up space in the context. For complex tasks, a thinking chain can take 2000-5000 tokens before the model even starts responding. If the context is too small, the reasoning is cut off or the answer is empty or truncated; if it's large, the KV cache grows, and the model slows down. Solution: for reasoning tasks, keep the context at least 4096 and disable thinking for simple queries (with the flag --think=false, the command /set nothink or the switch /no_think in Qwen3 models).

General rule: if ollama run takes longer than 30 seconds to load and the first response comes after a minute — the model is too large for your system. Don't expect it to "warm up" — close it and choose a smaller model.

Comparison of models by size, quality, and tasks — in the article Top 10 Ollama Models in 2026: Which to Choose.

Conclusion: A typical story looks like this: a person thinks a larger model will give a better result, downloads a 13-14B, waits a minute for the first response, and deletes it. After switching to 3-4B, performance immediately improves. On 8GB, a better strategy is to choose a model that works quickly and stably, rather than struggling with one that "almost fits." To the classic pitfalls, another has been added — thinking mode: new models think aloud and quietly consume your context before the first word of the response.

🎯 Estimated Speed: What to Expect in Practice

Short answer: On an 8GB M1 Mac, 3–4B models deliver around 15–28 tokens/sec, and 7–8B models deliver 8–12 tok/s. On a CPU-only Windows laptop, it's two to three times slower for larger models, but 3–4B models on modern CPUs achieve 8–12 tok/s — which is already sufficient for comfortable use. Below is a summary table for guidance.

Benchmarks found online are often conducted on a clean system without other software. In reality, with VS Code open and 5 Chrome tabs running — the numbers will be lower.

⚠️ Honest about the source of the numbers. These are not my own measurements, but a compilation from open tests (LocalLLM.in, SitePoint, LocalAIMaster). They should be treated as ranges for orientation. Below, I show how to measure the actual speed on your hardware in two minutes.

Performance Summary Table

Data is approximate, collected from LocalLLM.in, SitePoint, and LocalAIMaster. Actual speed depends on system load, context size, and background processes. For GPUs and Apple Silicon, enabling OLLAMA_FLASH_ATTENTION=1 might help with long contexts. Gemma 4 E4B is intentionally omitted from the table: its default tag (9.6 GB) does not fit on 8 GB.

📏 How to Measure Speed on Your Hardware

ollama run phi4-mini --verbose
# after the response, look for the line "eval rate: ... tokens/s" — this is the generation speed

ollama ps
# in the PROCESSOR column, it should say "100% GPU" (if you have a GPU) — otherwise, some layers are on the CPU

The eval rate line shows the actual response generation speed; prompt eval rate shows how quickly the model processes your query. Measure with the programs you typically have open (IDE, browser) for honest numbers.

What do these numbers mean in practice?

• ✔️ 15+ tok/s: comfortable interactive chat — the response appears faster than you can read it
• ✔️ 8–15 tok/s: usable, but you'll feel a delay on longer responses
• ⚠️ 3–6 tok/s: acceptable for one-off tasks (debugging, analysis), frustrating for active chat
• ❌ <3 tok/s: the model is too large for this system

Conclusion: For daily work on 8GB, aim for 3–4B models — they provide 15–28 tok/s on Apple Silicon and 8–12 tok/s on CPU, keeping the system responsive. Phi-4 Mini stands out with the best CPU performance in its class (~12 tok/s on a modern i7) — significantly better than Phi-3 Mini, which it replaced. 7–8B models (Qwen3 8B, DeepSeek R1 8B) are for specific tasks when you're ready to close everything else and wait.

❓ Frequently Asked Questions (FAQ)

Can I run Ollama on a laptop with 8GB of RAM?

Yes. Models with 1–4B parameters (Phi-4 Mini, Llama 3.2 3B, Qwen3 4B) run comfortably on any system with 8GB. 7–8B models run at the limit — you'll need to close unnecessary programs. More details in the Ollama Installation Guide.

What is the best model for 8GB of RAM?

It depends on the task. For code — Qwen3 4B (2.5 GB) or Qwen3.5 4B (3.4 GB). For text and chat — Llama 3.2 3B or Gemma 3 4B. For reasoning and debugging — DeepSeek R1 8B (at the 8GB limit) or Phi-4 Mini Reasoning (a lighter version with chain-of-thought). A single tool for everything — Qwen3.5 4B: multimodal, 256K context, thinking mode, but heavier than Qwen3 4B. A full model comparison is in the article Top 10 Ollama Models in 2026.

Do I need a GPU for Ollama?

No, Ollama also works on CPU. But with a GPU (discrete or Apple Silicon), the speed is many times higher. On a CPU-only system with 8GB, stick to 3–4B models for comfortable use. Phi-4 Mini is one of the best choices for CPU-only: according to third-party tests, it achieves about 12 tok/s on a modern i7 without a GPU.

What's better: a 7B model in Q2 or a 3B model in Q4?

Almost always — 3B in Q4. Aggressive quantization (Q2) significantly reduces the quality of responses, especially on complex tasks. A smaller model with normal quantization will yield better results. If Q4_K_M literally doesn't fit by a few hundred MB — try IQ4_XS for the same model, if such a tag exists on Hugging Face.

Do small models support Ukrainian?

It varies. Llama 3.2 officially supports 8 languages, Ukrainian is not among them. Qwen3 and Gemma claim support for 100+ languages, including Ukrainian. It's best to test the quality of Ukrainian in each model with your own texts: give a few typical prompts and compare the responses.

Can Ollama on 8GB replace ChatGPT?

For daily tasks — summarization, simple questions, code generation — yes. For basic multimodality (image analysis, screenshots) — also yes: Gemma 3 4B and Qwen3.5 4B accept images. For complex multi-step analysis, working with large contexts, and tasks requiring maximum accuracy — cloud models are still stronger. The optimal approach is hybrid: Ollama for regular tasks, ChatGPT/Claude for complex ones. More details in the article Ollama vs ChatGPT vs Claude: When Local AI is Better.

How much disk space is needed?

One 3–4B model in Q4 takes up approximately 2–3.5 GB on disk. Three models for different tasks — 7–9 GB. Ollama stores models in ~/.ollama; you can delete downloaded models with the command ollama rm model_name.

Is it worth upgrading to 16GB?

If you plan to work regularly with local AI — definitely yes. 16GB unlocks access to 9–14B models (Qwen3.5 9B at 6.6 GB, Qwen3 14B at 9.3 GB), 7B models in higher quality, and comfortable work with large context windows. Ollama's MLX backend is currently targeted at Macs with 32GB+, so 16GB won't unlock it. The difference in capabilities between 8GB and 16GB is the largest across the entire spectrum.

✅ Conclusions

8GB of RAM is not a death sentence for local AI, but it's a limit that requires informed decisions. Here's the main takeaway:

• ✔️ 3–4B models — the comfort zone: Phi-4 Mini, Llama 3.2 3B, Qwen3 4B, Qwen3.5 4B, Gemma 3 4B run fast and stable, leaving space for your IDE and browser
• ✔️ 7–8B models — the working zone: DeepSeek R1 8B, Qwen3 8B run at the limit (5.2 GB each), but provide noticeably better quality for specific tasks
• ✔️ Q4_K_M — the only sensible quantization choice on 8GB: a smaller model with Q4 is always better than a larger one with Q2
• ✔️ Apple Silicon with 8GB — the best budget option: unified memory offers an advantage over CPU-only systems
• ✔️ Don't bother trying: 13B+ models, Q8 quantization, running two models simultaneously, MoE with a large total size, and thinking mode without sufficient context
• ✔️ Look at the tag size, not the name: newer generations like Gemma 4 have heavier default tags, and they won't fit on 8GB

I personally use this exact approach: I keep several models for different tasks — one for code, another for text, a separate one for debugging. Each model has its strength, and instead of one large model that might not fit in memory, it's better to have 2–3 specialized lightweight ones. Switching between them via ollama run takes seconds.

If you're just starting — install Ollama using our guide, download phi4-mini, and give it a try. In five minutes, you'll have a working local AI — no subscriptions, no internet, no data sent externally.

And if you need a website, blog, or web application with integrated AI functionality — contact us at WebsCraft, we'll help you implement it.

📖 Sources

• Ollama Model Library — official registry of models and tag sizes
• Ollama — Official Documentation and FAQ — OLLAMA_CONTEXT_LENGTH, OLLAMA_FLASH_ATTENTION, OLLAMA_KV_CACHE_TYPE, OLLAMA_MAX_LOADED_MODELS
• Ollama Blog — MLX on Apple Silicon (March 2026)
• Switching to Mac — Ollama MLX: Memory Requirements
• SSD Nodes — Ollama KV cache quantization: q8_0 vs f16
• NeedToKnowIT — Ollama Models and Download Sizes — summary of tag sizes
• iTechGuides — Gemma 4 in Ollama: Tag Sizes
• LocalLLM.in — Ollama VRAM Requirements: Complete 2026 Guide
• LocalLLM.in — Best Local LLMs for 8GB VRAM
• SitePoint — Best Local LLM Models 2026: Developer Comparison
• LocalAIMaster — Best Ollama Models 2026: 15 Ranked
• LocalAIMaster — Best AI Models for 8GB RAM in 2026
• LocalAIMaster — Ollama RAM & VRAM Calculator: Every Model
• LocalAIMaster — Phi-4 Mini: Microsoft's Best Small Model (3.8B)
• StudyHUB — Best Ollama Models in 2026: Top 10 Ranked
• RunAIHome — Quantization Q4/Q5/Q6/Q8: Quality Loss in 2026
• Morph — Best Ollama Models: Ranked for Coding, RAG & Agents