Connect any OpenAI-compatible provider to Talon

Talon now supports an experimental openai-compatible provider. Point it at any endpoint that speaks the OpenAI chat API and Talon can run foreground agents and background workers through that endpoint without changing the rest of the daemon.

The obvious use case is Ollama. That includes local Ollama, Ollama Cloud, and any compatible gateway sitting in front of a model you want to use as a Talon provider. In practice, that means you can keep Talon’s existing persona system, tools, context rotation, background workers, and observability stack while swapping the execution backend underneath one or more personas.

Minimal configuration

agentRunner:
  defaultProvider: openai-compatible
  providers:
    openai-compatible:
      enabled: true
      command: node
      contextWindowTokens: 256000
      contextManagement:
        enabled: true
        triggerMetric: input_tokens
        thresholdRatio: 0.75
        recentMessageCount: 10
        summarizer: session-observer
      options:
        baseUrl: ${OLLAMA_BASE_URL}
        defaultModel: glm-5.1
        providerId: ollama

backgroundAgent:
  enabled: true
  defaultProvider: openai-compatible
  providers:
    openai-compatible:
      enabled: true
      command: node
      contextWindowTokens: 256000
      options:
        baseUrl: ${OLLAMA_BASE_URL}
        defaultModel: glm-5.1
        providerId: ollama

auth:
  providers:
    ollama:
      apiKey: ${OLLAMA_API_KEY}
      baseURL: ${OLLAMA_BASE_URL}

The important part is providerId. Talon uses it to look up auth.providers.<providerId> first, so an ollama auth block can be reused cleanly by the provider.

Why this matters

Before PR #179, Talon had three first-class execution backends: Claude Code, Gemini CLI, and Codex CLI. That covered a lot of ground, but it still left a hole for people who wanted Talon’s orchestration model with a self-hosted or third-party OpenAI-compatible endpoint.

This closes that gap. You can now route a persona through Ollama or another compatible API and still keep:

• persona-per-channel routing
• background workers
• capability-gated tools
• context rotation
• Langfuse tracing

Ollama Cloud, GLM-5.1, and observational memory

The configuration that keeps coming up in early testing is Ollama Cloud with glm-5.1, paired with Talon’s new observational-memory mode from PR #189. Instead of overwriting one rolling summary, observational memory appends dated decision-log entries and preserves the current task plus the next suggested continuation across context rotations.

That combination matters because it gives cheaper OpenAI-compatible models a stronger long-thread operating mode. With glm-5.1 on Ollama Cloud and Talon’s observational memory, users are already reporting a setup that rivals Claude Sonnet 4.6 for many agent tasks at a fraction of the cost. That is not a benchmark claim and it will depend heavily on the workload, but it is a credible reason to try the new provider if your main concern is price-performance on tool-using agents.

Two caveats

First, openai-compatible is still marked experimental. Talon’s wrapper is doing real work here to normalize streaming, tools, and usage reporting across endpoints that are only “compatible” in the broad sense.

Second, prompt-cache metrics depend on the upstream. Some OpenAI-compatible servers expose cached token counts cleanly. Ollama currently does not, so triggerMetric: input_tokens is the safer default there.

If you want the full configuration details, start with the Providers guide. That page now documents the openai-compatible provider, its auth lookup behavior, and the background-worker path.