How to Install Ollama and Chat With a Local Model in 20 Minutes

No GPU? Doesn't matter. No cloud account? Doesn't matter either. Here's how to get a real LLM running on your own machine with Ollama - the same steps we use when we test it for the catalog.

Step 1: Install Ollama

Go to ollama.com, download the installer for your OS, run it. That's genuinely it. On Windows it sits in the tray, on macOS it's a menu bar thing. I think there's a Linux one-liner too, something like curl -fsSL https://ollama.com/install.sh | sh, but don't quote me on the exact URL - just check the site.

Step 2: Pull a model

Open a terminal and run:

ollama pull llama3.1:8b

This downloads a few gigabytes, so go do something else for a bit. If your connection is slow, it will feel like forever. It's fine. Grab a coffee.

Step 3: Chat with it

Then run ollama run llama3.1:8b and just... type. It works right there in the terminal. The first reply is always slow because the model is loading - give it a few seconds before you assume it's broken.

Honestly, the first time I saw a model answer me on hardware that had never touched a GPU, I laughed out loud. It's not ChatGPT-fast, and it's not ChatGPT-smart. But it's yours. Offline. Private. Nobody's reading your prompts.

When it goes wrong

It will go wrong, so here's the short version. "Failed to pull" usually means disk space or a typo in the model name. Slow replies on Windows are often because Ollama didn't get access to your GPU - check the tray icon. And if the terminal looks frozen during a long reply, it's not frozen, it's thinking. Wait.

Make it nicer

Terminal chat is fine, but if you want a proper chat page, install Open WebUI on top. It connects to your local Ollama automatically and suddenly it feels like a real product. There's also a way to get a localhost chat UI without Docker, but honestly, Open WebUI is the reliable path.

Anyway - that's the whole setup. Twenty minutes, maybe thirty if the download drags. Go run something local.