Ollama: Since When Do Intel's Integrated Graphics Help?

I actually wanted to run a model with 744 billion parameters on my laptop (parameters are the learned numbers inside a model, roughly a measure of its size). That didn't happen. Instead I stumbled over something I hadn't seen coming: my laptop had been working with the handbrake on the whole time.

The plan: a giant model on a laptop

It started with colibri, an open-source project that is getting a lot of attention right now. Big models today often consist of many specialists, so-called experts (the technical term: Mixture of Experts). Only a few of them work on each word. So colibri keeps the fixed core and the experts currently in demand in memory and loads the rest from the SSD when needed.

According to the project, that works. Its own numbers are honest, though: on a normal laptop the flagship model manages 0.05 to 0.1 tokens per second from a cold start (a token is roughly a syllable). A paragraph of 100 tokens takes a quarter to half an hour.

Impressive technology. Useless for my day-to-day work. But one detail in the numbers stuck: on integrated graphics, a small model ran noticeably faster than on the processor.

Hang on. Integrated graphics?

Anyone getting into local AI reads the same thing everywhere: you need a graphics card. From Nvidia. Ideally with lots of memory. Everything else is tinkering.

My laptop has no Nvidia card. It has an Intel Core i5, 64 GB of RAM and the graphics that come built into the processor: an Intel Iris Xe. For months it has been running Ollama, the program I use for local models. Sluggishly, especially with long documents.

A look at docker exec ollama ollama ps said: 100% CPU. The graphics were on a break. As always.

And now there's suddenly more on offer? Huh. Since when?

What happened

For a long time, Ollama could use graphics chips from Nvidia, AMD and Apple. Intel wasn't on the list. Then Vulkan came along, an open interface for graphics chips from every vendor:

• October 2025: Vulkan as an experiment, only for people who compile Ollama themselves.
• November 2025: Vulkan ships in the ready-made packages, but has to be switched on by hand. Ollama's release notes explicitly mention Intel and integrated graphics.
• June 2026: Vulkan is on by default. Integrated graphics, however, have had a separate switch since then. Without it, Intel's graphics stay out.

That's the punchline. The support has been there for months. It just doesn't switch itself on. If you don't know, you won't notice.

The config to copy

The relevant part of my docker-compose.yml, tested with Ollama 0.40.8:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    environment:
      OLLAMA_CONTEXT_LENGTH: "131072"
      OLLAMA_IGPU_ENABLE: "1"
      OLLAMA_FLASH_ATTENTION: "1"
      OLLAMA_KV_CACHE_TYPE: "q8_0"
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - ollama-models:/root/.ollama

volumes:
  ollama-models:

What the lines do (the first three points are new):

• devices passes the graphics chip through to the Docker container.
• OLLAMA_IGPU_ENABLE is the switch for integrated graphics.
• OLLAMA_KV_CACHE_TYPE saves memory. For every token it reads, the model keeps intermediate results, the so-called KV cache. With q8_0 they are stored at half the size. This requires OLLAMA_FLASH_ATTENTION, a more economical way of calculating.
• OLLAMA_CONTEXT_LENGTH sets how much text the model may see at once. A large context costs memory.

Add port mapping and networking the way you usually do. Then docker compose up -d ollama, and ollama ps shows 100% GPU.

Almost.

The crash

The first test ended after about two minutes with device lost. The system log said: GPU HANG.

My reading: Ollama sends the text to the graphics chip in chunks of 512 tokens (the batch size). For the small chip, that's too much in one go. The Linux driver assumes it has crashed and resets it. A watchdog with good intentions.

With 128 tokens per chunk it ran stably. There is currently no environment variable for this; the value goes into the model:

docker exec ollama sh -c 'printf "FROM gemma3:12b\nPARAMETER num_batch 128\n" > /tmp/Modelfile && ollama create gemma3:12b -f /tmp/Modelfile'

That downloads nothing new. Run it once per chat model and repeat it after every ollama pull.

The result: astonishing

The test candidate was Gemma 3 (an open model from Google) with 12 billion parameters. Task: summarise a text of 6,761 tokens. Before: Ollama's defaults; after: the new configuration:

A factor of 4.5 for reading. From graphics that so far had only been driving the screen.

Is that really down to the graphics? Cross-check with a shorter text and identical settings, once without and once with the graphics: 6.7 versus 32.7 tokens per second for reading. Almost five times as fast. For writing, though, only just over 10% more. The writing gain in the table probably comes mainly from the memory settings.

And the stress test: a document of 23,683 tokens, read in 21.4 minutes, without a crash. Extrapolated from the CPU speed in the table, it would have taken around 80 minutes.

What it doesn't do

• The models don't get smarter. A small open model stays a small open model, with all its weaknesses. It just answers sooner.
• For writing, the graphics barely help. They share the memory with the CPU, and when writing, the speed of that memory is exactly the limit.
• Sleep mode aborts running requests. Ollama itself keeps running normally afterwards.
• I only tested Intel graphics on Linux. Whether Ollama uses AMD integrated graphics the same way, I haven't tried.

Is it worth it for you?

If you run Ollama in Docker on a machine without a dedicated graphics card: yes, try it. It costs nothing, you already have the hardware. To go back, delete devices and OLLAMA_IGPU_ENABLE again. Without Docker, you should be able to set the same environment variables on the Ollama service; I haven't tested that.

If you only ask short questions, you will notice less. The strength shows with long texts: contracts, minutes, manuals.

And the giant model? It stays on machines built for it. My laptop learned something new instead. Actually, it could have done it for months.