Model Context Protocol (MCP) is an open standard created by Anthropic that lets AI models connect to external tools and data sources — file systems, databases, web browsers, APIs — through a standardised interface. Combined with Ollama, MCP lets you build powerful local AI agents that can access real-world context without any cloud dependency.
What Is MCP?
MCP defines how an AI model communicates with external tools. An MCP server exposes capabilities (read a file, run a search, query a database), and an MCP client (your application or agent framework) connects the model to those capabilities. Anthropic open-sourced the protocol in late 2024, and it has since become a de facto standard adopted by most major AI tooling frameworks.
Think of it as USB for AI tools — a standard plug that works across different models and platforms.
Why Run MCP with Ollama?
Most MCP implementations assume you are using a cloud API (Claude, GPT-4). Running MCP with a local Ollama model means:
- Your files, database queries, and tool outputs never leave your machine
- No API costs for agent runs that make many tool calls
- Works on an air-gapped network with no internet access
- Full control over the model and its behaviour
What You Need
- Ollama running with a capable model — Llama 3.3 or Qwen2.5 7B+ recommended for tool calling
- Python 3.10+
- An MCP client library — the official Python SDK from Anthropic, or a framework like LangChain or LlamaIndex that supports MCP
Installing the MCP Python SDK
pip install mcp anthropicNote: you are installing the MCP SDK — you do not need an Anthropic API key to use MCP with Ollama.
Connecting Ollama to an MCP Server
Here is a minimal example connecting Ollama to the official MCP filesystem server, which lets the model read and write files:
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
import ollama
import asyncio
async def run_agent(prompt):
server_params = StdioServerParameters(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem", "/path/to/your/files"]
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
tools = await session.list_tools()
# Pass tools to Ollama
response = ollama.chat(
model='llama3.3',
messages=[{'role': 'user', 'content': prompt}],
tools=[tool.dict() for tool in tools.tools]
)
return response
asyncio.run(run_agent("List all the Python files in my project folder"))Popular MCP Servers
The MCP ecosystem has grown rapidly. Useful servers for local use:
- Filesystem — read/write local files.
@modelcontextprotocol/server-filesystem - SQLite — query local SQLite databases.
@modelcontextprotocol/server-sqlite - Brave Search — web search (needs a free Brave API key).
@modelcontextprotocol/server-brave-search - Git — read repository history and diffs.
@modelcontextprotocol/server-git - Fetch — retrieve content from URLs.
@modelcontextprotocol/server-fetch
Which Ollama Models Support Tool Calling?
MCP tool calling requires a model that supports function/tool calling. Confirmed working models:
- Llama 3.3 (8B and 70B)
- Llama 4 Scout
- Qwen2.5 (7B+)
- Qwen3-Coder
- Mistral (7B+)
Smaller models (under 4B) often struggle with reliable tool calling. Use at least a 7B model for agentic MCP workflows.
Using MCP with Open WebUI
Open WebUI has built-in MCP support. In the Open WebUI settings, add MCP server configurations and they become available as tools in the chat interface — no coding required. This is the easiest way to give your local Ollama models tool access.
Practical Use Cases
- Local file assistant — ask the model to find, summarise, or edit files on your machine
- Database Q&A — connect to SQLite and ask natural language questions about your data
- Research agent — combine Brave Search + Fetch + Filesystem to research and save notes locally
- Code review agent — connect to your Git repository and ask for a review of recent changes
