Local Agentic Programming on the Cheap: Claude Code + Ollama + Gemma4

# macOS and Linux -- one-line install
curl -fsSL https://ollama.com/install.sh | sh

# Verify version -- must be 0.14.0+ for Anthropic Messages API support
# The Anthropic-compatible endpoint was added in January 2026
ollama version
# Expected: ollama version is 0.22.x or higher (as of May 2026)

# Windows: download the native installer from https://ollama.com
# WSL2 is recommended if you want GPU passthrough on Windows

curl http://localhost:11434
# Expected response: Ollama is running

# The 26B MoE -- recommended for this setup (~18 GB download)
ollama pull gemma4:26b

# While you wait, confirm the download is progressing
ollama ps
# Shows currently downloading or running models

# Optional: also pull the 31B for comparison on capable hardware
ollama pull gemma4:31b

# Confirm the pull completed
ollama list
# Should show gemma4:26b with size and modification date

# Prerequisites: Node.js 18 or later
node --version   # Confirm you are on 18+

# Install Claude Code CLI globally
npm install -g @anthropic-ai/claude-code

# Verify the install
claude --version

# ~/.ollama/Modelfiles/gemma4-claude
# Gemma 4 26B MoE variant tuned for Claude Code agentic sessions.
# Bakes context window, temperature, and system prompt into the model
# so every Claude Code session starts with the correct configuration.
#
# Build with:
#   mkdir -p ~/.ollama/Modelfiles
#   ollama create gemma4-claude -f ~/.ollama/Modelfiles/gemma4-claude

FROM gemma4:26b

# Context window -- 65536 tokens (64K) is the tested-safe floor for real
# codebases without triggering swap on 16-18 GB VRAM systems.
# Increase to 131072 (128K) if you have headroom on 24 GB+ systems.
# Do not go above 131072 unless you have profiled your memory usage
# under load -- Ollama pre-allocates the full KV cache upfront.
PARAMETER num_ctx 65536

# Temperature -- 0.2 is deliberately low for agentic coding.
# Higher temperature introduces variability in tool call parameter
# formatting that causes Claude Code's tool validator to reject calls.
# For creative tasks, you would set this higher. For agentic loops: low.
PARAMETER temperature 0.2

# top_p -- nucleus sampling threshold. 0.9 keeps generation focused
# while avoiding the repetition loops that top_p=1.0 can produce on
# long agentic sessions.
PARAMETER top_p 0.9

# repeat_penalty -- penalizes the model for repeating tokens.
# 1.15 helps prevent tool call loops where Gemma 4 retries the same
# failed tool call with nearly identical parameters indefinitely.
PARAMETER repeat_penalty 1.15

# num_predict -- maximum tokens per response. 4096 is sufficient for
# most code patches. Increase to 8192 if you regularly generate
# large files in a single generation.
PARAMETER num_predict 4096

# System prompt -- reinforces coding agent behavior and explicit
# tool use discipline. Gemma 4 benefits from being reminded to
# commit to tool calls rather than describing what it would do.
SYSTEM """You are a senior software engineer operating as a coding agent.

When working with code:
- Read files before editing them. Never assume file contents.
- Make one focused change at a time and verify it before proceeding.
- When a tool call fails, examine the error carefully before retrying.
  Do not retry with identical parameters. Diagnose first.
- Prefer surgical edits over full file rewrites.
- Run tests after each meaningful change, not after a batch of changes.
- If you are uncertain about the codebase structure, read more files
  rather than guessing.

Be precise and methodical. Avoid explaining what you are about to do
when you could simply do it."""

# Create the Modelfiles directory if it does not exist
mkdir -p ~/.ollama/Modelfiles

# Save the Modelfile content from above to this path, then build:
ollama create gemma4-claude -f ~/.ollama/Modelfiles/gemma4-claude

# Verify the variant was created
ollama list
# Should show gemma4-claude alongside gemma4:26b

# Quick smoke test -- verify it loads and responds
ollama run gemma4-claude "What is the time complexity of binary search and why?"
# Expect a clear, concise technical response within a few seconds

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:11434",

    "ANTHROPIC_AUTH_TOKEN": "ollama",

    "ANTHROPIC_API_KEY": "",

    "ANTHROPIC_MODEL": "gemma4-claude",

    "ANTHROPIC_DEFAULT_SONNET_MODEL": "gemma4-claude",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "gemma4-claude",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "gemma4-claude",

    "CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS": "1"
  }
}

# In your project root
mkdir -p .claude

cat > .claude/settings.json << 'EOF'
{
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:11434",
    "ANTHROPIC_AUTH_TOKEN": "ollama",
    "ANTHROPIC_API_KEY": "",
    "ANTHROPIC_MODEL": "gemma4-claude",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "gemma4-claude",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "gemma4-claude",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "gemma4-claude",
    "CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS": "1"
  }
}
EOF

pip install httpx   # Async HTTP client for the verification script

#!/usr/bin/env python3
"""
verify_local_setup.py

Verifies the full Claude Code + Ollama + Gemma 4 stack before use.
Runs three checks in sequence:
  1. Ollama health and model availability
  2. Basic Anthropic Messages API call
  3. Tool calling round-trip

Prerequisites:
  pip install httpx

How to run:
  python verify_local_setup.py

Expected output on a working setup:
  [PASS] Ollama is running on localhost:11434
  [PASS] Model 'gemma4-claude' is available
  [PASS] Anthropic Messages API call successful
  [PASS] Tool calling: model produced a valid tool_use block
  All checks passed -- Claude Code + Ollama + Gemma 4 is ready.
"""

import httpx
import json
import sys

# ── Configuration ─────────────────────────────────────────────────────────────
OLLAMA_BASE_URL = "http://localhost:11434"
MODEL_NAME      = "gemma4-claude"   # Must match your Modelfile variant name
TIMEOUT         = 120.0             # Seconds -- generation can be slow on first call


def check_ollama_health() -> bool:
    """
    Check 1: Verify Ollama is running and responding.
    Hits the root endpoint which returns 'Ollama is running' when healthy.
    """
    print("\nCheck 1: Ollama health")
    try:
        response = httpx.get(OLLAMA_BASE_URL, timeout=5.0)
        if "Ollama is running" in response.text:
            print(f"  [PASS] Ollama is running on {OLLAMA_BASE_URL}")
            return True
        else:
            print(f"  [FAIL] Unexpected response: {response.text[:100]}")
            return False
    except httpx.ConnectError:
        print(f"  [FAIL] Cannot connect to {OLLAMA_BASE_URL}")
        print("         Is Ollama running? Try: ollama serve")
        return False


def check_model_available() -> bool:
    """
    Check 2: Verify the specific model variant is available in Ollama.
    Uses the /api/tags endpoint which lists all pulled models.
    """
    print("\nCheck 2: Model availability")
    try:
        response = httpx.get(f"{OLLAMA_BASE_URL}/api/tags", timeout=5.0)
        data     = response.json()
        models   = [m["name"] for m in data.get("models", [])]

        # Normalize: Ollama may add ":latest" if not specified
        normalized = [m.split(":")[0] for m in models]

        if MODEL_NAME in models or MODEL_NAME in normalized:
            print(f"  [PASS] Model '{MODEL_NAME}' is available")
            return True
        else:
            print(f"  [FAIL] Model '{MODEL_NAME}' not found")
            print(f"         Available models: {', '.join(models) or 'none'}")
            print(f"         Run: ollama create {MODEL_NAME} -f ~/.ollama/Modelfiles/gemma4-claude")
            return False
    except Exception as e:
        print(f"  [FAIL] Error checking model list: {e}")
        return False


def check_messages_api() -> bool:
    """
    Check 3: Send a basic Anthropic Messages API call to the local endpoint.
    Verifies the request format, model routing, and basic generation work.
    Uses the same /v1/messages path and request schema that Claude Code uses.
    Note: Claude Code uses http://localhost:11434 (root), not /v1.
    The Anthropic-compatible API is at /api/chat or the root -- Ollama routes it.
    """
    print("\nCheck 3: Anthropic Messages API call")

    payload = {
        "model": MODEL_NAME,
        "max_tokens": 100,
        "messages": [
            {
                "role": "user",
                "content": "Reply with exactly: VERIFICATION_OK"
            }
        ]
    }

    headers = {
        "Content-Type":      "application/json",
        "x-api-key":         "ollama",            # Required by the API spec; value ignored locally
        "anthropic-version": "2023-06-01"         # Required version header
    }

    try:
        response = httpx.post(
            f"{OLLAMA_BASE_URL}/v1/messages",
            json=payload,
            headers=headers,
            timeout=TIMEOUT
        )

        if response.status_code != 200:
            print(f"  [FAIL] HTTP {response.status_code}: {response.text[:200]}")
            return False

        data = response.json()

        # Anthropic Messages API response structure:
        # { "content": [{"type": "text", "text": "..."}], "stop_reason": "..." }
        content_blocks = data.get("content", [])
        text_blocks    = [b for b in content_blocks if b.get("type") == "text"]

        if not text_blocks:
            print(f"  [FAIL] No text content in response: {json.dumps(data, indent=2)}")
            return False

        response_text = text_blocks[0].get("text", "")
        print(f"  [PASS] Anthropic Messages API call successful")
        print(f"         Model response: {response_text[:80]}")
        return True

    except Exception as e:
        print(f"  [FAIL] Request failed: {e}")
        return False


def check_tool_calling() -> bool:
    """
    Check 4: Verify tool calling works end-to-end.
    This is the most important check for Claude Code agentic use.
    Claude Code relies on the model correctly producing tool_use blocks
    for every file operation, shell command, and code execution.

    Sends a simple tool definition and a prompt that should trigger it.
    Verifies the model returns a tool_use block (not just text describing the call).
    """
    print("\nCheck 4: Tool calling verification")

    # A minimal tool definition using the Anthropic function calling schema
    tools = [
        {
            "name": "read_file",
            "description": "Read the contents of a file at the given path.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "path": {
                        "type": "string",
                        "description": "The absolute or relative file path to read"
                    }
                },
                "required": ["path"]
            }
        }
    ]

    payload = {
        "model": MODEL_NAME,
        "max_tokens": 256,
        "tools": tools,
        # Force the model to call a tool rather than respond in text.
        # tool_choice: {"type": "any"} requires any tool use.
        # Remove this if testing whether the model self-selects tools.
        "tool_choice": {"type": "any"},
        "messages": [
            {
                "role": "user",
                "content": "Read the file at /tmp/test.py and show me its contents."
            }
        ]
    }

    headers = {
        "Content-Type":      "application/json",
        "x-api-key":         "ollama",
        "anthropic-version": "2023-06-01"
    }

    try:
        response = httpx.post(
            f"{OLLAMA_BASE_URL}/v1/messages",
            json=payload,
            headers=headers,
            timeout=TIMEOUT
        )

        if response.status_code != 200:
            print(f"  [FAIL] HTTP {response.status_code}: {response.text[:200]}")
            return False

        data           = response.json()
        content_blocks = data.get("content", [])
        tool_blocks    = [b for b in content_blocks if b.get("type") == "tool_use"]

        if not tool_blocks:
            print("  [FAIL] Model did not produce a tool_use block")
            print("         This means tool calling is not working correctly.")
            print("         Agentic Claude Code sessions will fail on file operations.")
            print(f"         Full response: {json.dumps(data, indent=2)}")
            return False

        tool_call  = tool_blocks[0]
        tool_name  = tool_call.get("name", "")
        tool_input = tool_call.get("input", {})

        print(f"  [PASS] Tool calling: model produced a valid tool_use block")
        print(f"         Tool called: {tool_name}")
        print(f"         Parameters:  {json.dumps(tool_input)}")

        # Sanity check: did it call the right tool with the right parameter?
        if tool_name == "read_file" and "path" in tool_input:
            print(f"         Tool name and parameter are correct.")
        else:
            print(f"         WARNING: Unexpected tool name or missing 'path' parameter.")
            print(f"         The model called a tool but not the expected one.")

        return True

    except Exception as e:
        print(f"  [FAIL] Request failed: {e}")
        return False


def main():
    print("=" * 60)
    print("Claude Code + Ollama + Gemma 4 Setup Verification")
    print("=" * 60)

    checks = [
        check_ollama_health,
        check_model_available,
        check_messages_api,
        check_tool_calling,
    ]

    results = [check() for check in checks]

    print("\n" + "=" * 60)
    passed = sum(results)
    total  = len(results)

    if all(results):
        print(f"All {total} checks passed.")
        print("Claude Code + Ollama + Gemma 4 is ready.")
        print(f"\nLaunch with: claude")
        sys.exit(0)
    else:
        failed_checks = [i + 1 for i, r in enumerate(results) if not r]
        print(f"{passed}/{total} checks passed. Failed: {failed_checks}")
        print("Resolve the failures above before using Claude Code locally.")
        sys.exit(1)


if __name__ == "__main__":
    main()

pip install httpx
python verify_local_setup.py

# Navigate to a project directory
cd ~/projects/my-service

# Confirm Claude Code sees the local configuration
claude --version
# Verify it does not prompt for an Anthropic API key -- if it does,
# the settings.json is not being read correctly

# Start an agentic session
claude

# Inside Claude Code, give the agent a concrete task:
# > Analyze the UserService class in src/user_service.py.
# > Write a pytest test suite covering all public methods.
# > Run the tests and fix any failures.
# > The goal is a clean pytest run with no skips.

→ read_file("src/user_service.py")
  Reading 247 lines...

→ list_files("src/")
  Found: user_service.py, models.py, db.py, exceptions.py

→ read_file("src/models.py")
  Reading 89 lines...

→ write_file("tests/test_user_service.py", [test content])
  Written: 312 lines

→ bash("python -m pytest tests/test_user_service.py -v 2>&1")
  Running 14 tests...
  FAILED tests/test_user_service.py::test_update_email_invalid
    AssertionError: Expected ValidationError, got None

→ read_file("src/user_service.py")  [targeted re-read of update_email method]
  ...

→ write_file("tests/test_user_service.py", [corrected test])
  Patched test_update_email_invalid assertion

→ bash("python -m pytest tests/test_user_service.py -v 2>&1")
  14 passed in 1.23s