Ollama (Local)

Ollama provides fully local LLM inference with no API key required and no data leaving your machine.

Prerequisites

Install Ollama:

# macOS / Linux
curl -fsSL https://ollama.ai/install.sh | sh

# Start the server
ollama serve

Setup

Default Configuration

No API key is needed. The default Ollama URL is http://localhost:11434.

Configuration File

[llm]
default_provider = "ollama"

[llm.ollama]
model = "llama3.2"
base_url = "http://localhost:11434"

Environment Variable (Remote Server)

export OLLAMA_BASE_URL="http://your-server:11434"

Available Models

Pull models from the Ollama registry:

ollama pull llama3.2
ollama pull codellama
ollama pull mistral
ollama pull deepseek-coder

Usage

# Default (llama3.2)
clawdius chat "explain this code" --provider ollama

# Specific model
clawdius chat "write tests" --provider ollama --model codellama

# Manage models via CLI
clawdius models list
clawdius models pull llama3.2
clawdius models health
clawdius models current

Managing Models

# List available local models
clawdius models list

# Pull a new model
clawdius models pull mistral

# Check server health
clawdius models health

# Custom server
clawdius models list --host 192.168.1.100 --port 11434

Configuration Reference

[llm.ollama]
model = "llama3.2"                       # Model to use
base_url = "http://localhost:11434"       # Ollama server URL

Connectivity Check

The setup wizard checks Ollama connectivity:

clawdius setup --provider ollama

If the server is unreachable, verify:

curl http://localhost:11434/api/tags

Performance Tips

• Use models with fewer parameters for faster responses
• Ensure sufficient RAM (at least 8GB for 7B models)
• Use llama3.2 for the best speed/quality tradeoff on consumer hardware
• Consider GPU acceleration for larger models

Troubleshooting

"Connection refused"

# Verify Ollama is running
ollama serve
curl http://localhost:11434/api/tags

"Model not found"

ollama pull llama3.2
clawdius models list