How to Use Ollama with Python: Library, OpenAI SDK, and REST API

The Ollama Python library is the official client for interacting with Ollama from Python code, but it isn’t the only way to call it — you can also use the OpenAI SDK for drop-in compatibility, or make direct HTTP requests with no dependencies at all. This guide covers all three approaches, plus streaming, async support, and embeddings, so you can pick whichever fits your project.

Prerequisites

Before you can call Ollama from Python, Ollama itself must be installed and running on your machine. If you have not already installed Ollama, download it from ollama.com and follow the setup instructions for your platform.

Once installed, start the Ollama server:

ollama serve

On macOS, Ollama runs automatically in the background after installation. On Linux, you may need to start it manually or configure it as a system service. The server listens on http://localhost:11434 by default.

You will also need at least one model pulled:

ollama pull llama3.2

Method 1: The Official Ollama Python Library

The simplest and most common option:

pip install ollama

This installs the official library published by the Ollama team. It requires Python 3.8 or later and has minimal dependencies. Import it with:

import ollama

Generating Text with generate()

The ollama.generate() function is the simplest entry point. It takes a model name and a prompt, sends a single-turn request to the local Ollama server, and returns a response dictionary.

import ollama

response = ollama.generate(model='llama3.2', prompt='Why is the sky blue?')
print(response['response'])

You can pass additional parameters using the options argument:

response = ollama.generate(
    model='llama3.2',
    prompt='Explain quantum entanglement simply.',
    options={
        'temperature': 0.7,
        'num_ctx': 4096
    }
)
print(response['response'])

Common options:

• temperature — Controls randomness. Lower values (e.g. 0.2) produce more deterministic output; higher values (e.g. 1.0) increase creativity.
• num_ctx — Context window size in tokens. Increase this for longer inputs.
• top_p — Nucleus sampling threshold.
• num_predict — Maximum tokens to generate.

Multi-Turn Conversations with chat()

For conversational applications where the model needs to remember prior messages, use ollama.chat(). Pass a list of message dictionaries with role and content keys. Roles are 'user', 'assistant', and 'system'.

import ollama

messages = [
    {'role': 'system', 'content': 'You are a helpful assistant who speaks concisely.'},
    {'role': 'user', 'content': 'What is the capital of France?'},
]

response = ollama.chat(model='llama3.2', messages=messages)
print(response['message']['content'])

To continue the conversation, append the assistant’s reply and the next user message, then call chat() again:

messages.append(response['message'])
messages.append({'role': 'user', 'content': 'And what is its population?'})

response = ollama.chat(model='llama3.2', messages=messages)
print(response['message']['content'])

Building a Simple Chatbot Loop

import ollama

messages = []

print("Chat with llama3.2. Type 'quit' to exit.")

while True:
    user_input = input("You: ").strip()
    if user_input.lower() == 'quit':
        break

    messages.append({'role': 'user', 'content': user_input})

    response = ollama.chat(model='llama3.2', messages=messages)
    reply = response['message']['content']

    messages.append({'role': 'assistant', 'content': reply})
    print(f"Assistant: {reply}")

Streaming Responses

By default, the library waits for the model to finish generating before returning anything. Passing stream=True returns a generator, allowing you to print tokens as they arrive:

import ollama

stream = ollama.chat(
    model='llama3.2',
    messages=[{'role': 'user', 'content': 'Tell me a short story about a robot.'}],
    stream=True
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

print()  # newline after stream ends

Streaming works with generate() too — each chunk has a 'response' key rather than 'message':

stream = ollama.generate(model='llama3.2', prompt='Count to ten.', stream=True)

for chunk in stream:
    print(chunk['response'], end='', flush=True)

Async Support

For async applications — FastAPI, asyncio — the library provides an AsyncClient that mirrors the synchronous API exactly:

import asyncio
from ollama import AsyncClient

async def main():
    client = AsyncClient()
    response = await client.chat(
        model='llama3.2',
        messages=[{'role': 'user', 'content': 'What is asyncio?'}]
    )
    print(response['message']['content'])

asyncio.run(main())

Method 2: Using the OpenAI Python SDK

If your project already uses the OpenAI SDK, you can point it at your local Ollama instance with one change — the base_url. This is ideal for swapping between local Ollama models and OpenAI’s API without changing application code:

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1',
    api_key='ollama'  # required but ignored by Ollama
)

response = client.chat.completions.create(
    model='llama3.2',
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Summarise the key features of Python 3.12.'}
    ]
)

print(response.choices[0].message.content)

Method 3: Direct HTTP Requests

No dependencies beyond requests — useful if you don’t want to add the ollama or openai packages to a project:

import requests

response = requests.post(
    'http://localhost:11434/api/chat',
    json={
        'model': 'llama3.2',
        'messages': [{'role': 'user', 'content': 'What is machine learning?'}],
        'stream': False
    }
)

data = response.json()
print(data['message']['content'])

Managing Models

Listing Available Models

result = ollama.list()
for model in result['models']:
    print(model['name'])

Pulling a Model

ollama.pull('llama3.2')

Stream pull progress:

for progress in ollama.pull('llama3.2', stream=True):
    status = progress.get('status', '')
    completed = progress.get('completed', 0)
    total = progress.get('total', 0)
    if total:
        print(f"{status}: {completed}/{total}", end='r')
    else:
        print(status)

Showing Model Information

info = ollama.show('llama3.2')
print(info)

Generating Embeddings

Embeddings convert text into a numerical vector that captures its semantic meaning — used for similarity search, retrieval-augmented generation (RAG), and clustering. You need a model designed for embeddings such as nomic-embed-text:

ollama pull nomic-embed-text

result = ollama.embeddings(model='nomic-embed-text', prompt='The quick brown fox')
vector = result['embedding']
print(f"Embedding dimension: {len(vector)}")

A minimal similarity example using cosine distance:

def get_embedding(text):
    result = ollama.embeddings(model='nomic-embed-text', prompt=text)
    return result['embedding']

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    mag_a = sum(x ** 2 for x in a) ** 0.5
    mag_b = sum(x ** 2 for x in b) ** 0.5
    return dot / (mag_a * mag_b)

query = "What is machine learning?"
candidates = [
    "Machine learning is a branch of artificial intelligence.",
    "The weather in London is often cloudy.",
]

query_vec = get_embedding(query)
for candidate in candidates:
    score = cosine_similarity(query_vec, get_embedding(candidate))
    print(f"Score {score:.4f}: {candidate}")

Connecting to a Remote Ollama Instance

If Ollama runs on a different machine, point the client at a custom host using ollama.Client:

client = ollama.Client(host='http://192.168.1.50:11434')

response = client.chat(
    model='llama3.2',
    messages=[{'role': 'user', 'content': 'Hello from a remote client!'}]
)
print(response['message']['content'])

The Client class supports all the same methods as the module-level functions. The async equivalent is AsyncClient(host='...'). See our guide to accessing Ollama remotely over Tailscale for setting up a remote host securely.

Error Handling

from ollama import ResponseError

try:
    response = ollama.chat(
        model='llama3.2',
        messages=[{'role': 'user', 'content': 'Hello'}]
    )
    print(response['message']['content'])
except ResponseError as e:
    print(f"Ollama error: {e.status_code} — {e.error}")
except Exception as e:
    print(f"Unexpected error: {e}")

The ResponseError object exposes a status_code (HTTP status) and an error string describing the problem. Common causes include referencing a model that has not been pulled, or attempting to connect when the Ollama server is not running.

Choosing the Right Model for Your Python Project

The best model depends on your use case:

• Coding assistance: See best Ollama models for coding
• Summarisation: See best Ollama models for summarisation
• RAG and embeddings: See best Ollama models for RAG

Quick Reference

• Install with pip install ollama and ensure ollama serve is running first
• Use ollama.generate() for single-turn completions, ollama.chat() for multi-turn conversations
• Pass stream=True to either function to receive tokens incrementally
• Use AsyncClient in async codebases — the API is identical but uses await
• Already using the OpenAI SDK? Point base_url at http://localhost:11434/v1 instead of switching libraries
• No dependencies at all? Call the REST API directly with requests
• Manage models with ollama.list(), ollama.pull(), and ollama.show()
• Generate semantic vectors with ollama.embeddings() using a dedicated embedding model
• Point to a remote instance using ollama.Client(host='http://...')
• Catch ResponseError for structured error handling

Next Steps

Once you’re comfortable calling Ollama from Python, the natural next step is building a RAG pipeline with LangChain or integrating Ollama into a web app using FastAPI. You can also explore the Docker deployment guide for running Ollama in a containerised environment.

---

Related Reading

• How to Use Ollama with LangChain
• Best Ollama Models for RAG