OpenAI-compatible provider and provider registry

Overview

Prove the phase-3 abstraction by implementing a second adapter. The OpenAI-compatible provider synthesizes block framing that Anthropic gives natively: BlockStart on the first tool-call delta carrying id and name, tool arguments accumulated by Index, and BlockStop for every open block at finish_reason. Add a provider registry with an injected CredentialLookup seam, per-agent provider and model selection, and capability validation at agent save time. One base URL covers OpenAI, Ollama, OpenRouter, vLLM and Groq.

Requirements

Functional

• internal/provider/openai implements provider.Provider over openai-go/v3 Chat Completions (not the Responses API: hango owns session state).
• Event synthesis: accumulate Choices[0].Delta.ToolCalls[i] by Index; emit BlockStart when a tool call first presents ID+Name, ToolInputDelta for each argument fragment, BlockStop for all open blocks at finish_reason; emit MessageStop with mapped stop reason and usage. Do not use ChatCompletionAccumulator (documented unsafe with parallel tool calls).
• Stop reason mapping: stop→EndTurn, length→MaxTokens, tool_calls→ToolUse, content_filter→Refusal; context-length errors map to ErrContextOverflow.
• Thinking blocks are unsupported: the adapter drops them from outgoing messages and reports the capability as absent, so a thinking-enabled agent on an OpenAI provider fails validation at agent save time rather than mid-run.
• Registry: `Get(ctx, providerName)` calls the injected `CredentialLookup`; in this phase the only implementation reads operator-set server config. An unknown provider or a provider with no configured credential returns a typed error surfaced as a 422 at agent save time and at run creation, never a 500 mid-run.
• The operator-set `HANGO_OPENAI_BASE_URL` is validated at startup: scheme must be `https`, or `http` only for a loopback host. A bad value fails startup with the other config errors. (Tenant-supplied base URLs, which need a full SSRF allowlist, arrive with the deferred BYO-credentials work.)
• Agents select provider and model; validation rejects a provider with no resolvable credentials at save time.
• `Capabilities` carries `Thinking` and `PromptCaching` only. `PromptCaching` is consumed by `PromptCacheMiddleware` (phase 10), which skips breakpoint placement when it is false. `ParallelTools` is deleted: tool batches execute sequentially in v1 (phase 9), so nothing would read it.

Non-functional

• Credentials exist only inside the registry at client construction: never logged, never returned by any endpoint, never placed on a span.
• The registry is process-global and stateless with respect to tenants: it holds built clients keyed by the resolved credential’s identity, and CredentialLookup is consulted per resolution. There is no tenant-keyed cache to bleed across tenants when BYO credentials land.
• The adapter is tested against recorded chunk fixtures including parallel tool calls with interleaved argument fragments.
• The same loop tests that pass against the Anthropic fake must pass against the OpenAI fake with no loop changes: that equivalence is the phase’s real deliverable.

Architecture

// internal/provider/registry.go
type Registry struct {
	creds CredentialLookup // injected; today server config, later a store. Red team: D1
	build map[string]Factory
}

type CredentialLookup func(ctx context.Context, providerName string) (Credential, bool, error)

type Credential struct {
	APIKey       string
	BaseURL      string
	DefaultModel string
}

type Factory func(Credential) (Provider, error)

func (r *Registry) Get(ctx context.Context, name string) (Provider, error) // lookup -> build -> ErrNoProvider

// internal/provider/openai/stream.go — the synthesis core
type blockAccum struct {
	index   int
	id      string
	name    string
	argsBuf strings.Builder
	started bool
}
// on each chunk: for every ToolCalls[i] delta -> ensure accum by Index, emit BlockStart once id+name known,
// emit ToolInputDelta for Arguments fragments; on finish_reason -> emit BlockStop for all open accums,
// then MessageStop with mapped StopReason and Usage.

// internal/provider/capability.go
type Capabilities struct {
	Thinking      bool // consumed by agent save-time validation
	PromptCaching bool // consumed by PromptCacheMiddleware (phase 10)
	// Red team: F15 — ParallelTools deleted: nothing consumes it (batches run sequentially in v1).
}

type CapabilityAware interface{ Capabilities() Capabilities }

Instrumentation

• The shared internal/provider/trace.go helper means the OpenAI adapter emits the identical chat {model} span shape with gen_ai.provider.name=openai (and gen_ai.system=openai), so all dashboards and Langfuse views work unchanged for the second provider. A test asserts attribute parity between adapters for an equivalent scripted stream.
• Adapter-specific span attributes: hango.provider.base_url_host (host only, never credentials), hango.provider.synthesized_blocks count, hango.provider.parallel_tool_calls.
• Registry spans: provider.resolve INTERNAL span with gen_ai.provider.name and hango.provider.credential_source (server_config today; the attribute exists so the BYO follow-up can add tenant without a dashboard change).
• Metrics: existing GenAI metrics gain the provider attribute (already present), plus hango.provider.resolutions counter (attributes provider, source, error.type).
• Log fields: credential resolution logs the provider, the source and the base-URL host — never the key or a full URL with query string.

Related Code Files

Create

• internal/provider/openai/provider.go, internal/provider/openai/convert.go, internal/provider/openai/stream.go, internal/provider/openai/stream_test.go, internal/provider/openai/testdata/*.json.
• internal/provider/registry.go, internal/provider/registry_test.go, internal/provider/capability.go.
• internal/provider/parity_test.go — drives the same scripted conversation through both adapters, pushing events into one provider.Accumulator per adapter, and asserts identical normalized event sequences and accumulated messages (modulo provider name and thinking).
• docs/learning/12-second-provider.md.
• No `internal/auth/crypt.go`, no credential store, no credential handlers, no migration.

Modify

• internal/provider/provider.go — document the synthesis contract for adapters lacking native block framing.
• internal/app/app.go — build the registry with the config-backed CredentialLookup.
• internal/run/worker.go — resolve the provider per run from the registry using the agent’s provider field.
• internal/server/handlers_agents.go — validate provider and model at save time, including the thinking capability check.
• internal/agent/mwcache.go — skip breakpoint placement when Capabilities.PromptCaching is false.
• internal/config/config.go — HANGO_OPENAI_API_KEY, HANGO_OPENAI_BASE_URL (validated https-or-loopback at startup).
• docs/system-architecture.md, docs/learning/README.md, .env.example.

Implementation Steps

1. Add github.com/openai/openai-go/v3, pinned exactly (the module releases near-daily).
2. Write internal/provider/capability.go (Thinking, PromptCaching) and implement CapabilityAware on both adapters; wire PromptCaching into PromptCacheMiddleware.
3. Write internal/provider/openai/convert.go: hango messages → ChatCompletionMessageParamUnion (tool results become tool role messages keyed by tool_call_id; system blocks become a system message; thinking blocks are dropped with a debug log), hango tool defs → ChatCompletionToolParam with the JSON Schema.
4. Write internal/provider/openai/stream.go implementing the synthesis exactly as sketched: a map[int]*blockAccum, text deltas as index 0 by convention, BlockStart emitted once per tool call as soon as both id and name are known, argument fragments emitted as ToolInputDelta with the block index, all open blocks closed at finish_reason, then MessageStop. Do not call ChatCompletionAccumulator.
5. Record fixtures for: text only, single tool call, two parallel tool calls with interleaved argument fragments, a tool call whose name arrives in a later chunk than its id, length finish, and a context-length error. Golden-test the produced event sequences and the accumulated message.
6. Write internal/provider/registry.go with the injected CredentialLookup (keeping provider free of store), the factory map, and typed ErrNoProvider/ErrProviderMisconfigured.
7. Wire internal/app to build the registry with the config-backed lookup, validating HANGO_OPENAI_BASE_URL as https-or-loopback at startup. The worker resolves the provider per run; resolution failures fail the run with a clear error before the first model call.
8. Write internal/provider/parity_test.go: drive the same scripted conversation (text, one tool call, parallel tool calls) through both fakes, accumulate each through a provider.Accumulator, and assert the normalized event sequences and accumulated messages match; then run the phase-5 loop test suite parameterized over both adapters.
9. Add validation in the agent handler: saving an agent whose provider lacks credentials, or which enables thinking on a provider without the capability, returns 422 with a specific message.
10. Add the instrumentation and the attribute-parity test.
11. Update docs and write the learning doc.

Learning Doc Outline

docs/learning/12-second-provider.md must contain:

• Concept: an abstraction is only proven by its second implementation; what “synthesizing block framing” means concretely.
• The wire-level asymmetry: Anthropic’s content_block_start/input_json_delta/content_block_stop versus OpenAI’s Delta.ToolCalls[] with Index, an id and name that appear only in the first fragment, and Arguments as partial JSON strings. Include an annotated chunk transcript next to the phase-3 Anthropic transcript.
• Why ChatCompletionAccumulator.JustFinishedToolCall is unsafe with parallel tool calls and hango accumulates by Index itself.
• Why Chat Completions and not the Responses API: hango owns session state, and Chat Completions is what compatible providers implement.
• Design choices: registry with an injected credential lookup, operator-set credentials in v1, capability interfaces for thinking and caching, save-time validation instead of mid-run failure.
• Why BYO tenant credentials were deferred: the contract asks for per-tenant API keys and quotas, not per-tenant provider keys, and encryption plus rotation plus an SSRF-safe tenant `base_url` is a security surface worth its own phase. Record what the seam preserves and what the follow-up must add.
• Trade-offs: seam-now-implementation-later vs building the table up front; failing at save time vs degrading at run time.
• What phase 3 got right and what had to bend: record honestly any normalized-model change this phase required.
• goclaw comparison: AdapterRegistry name→factory, Registry keyed "tenantID/name" with master-tenant fallback and an injected tenantFromCtx to avoid an import cycle, plus failover/cooldown/model-fallback machinery hango omits.
• deepagents comparison: provider abstraction is LangChain’s; the OpenAI/Anthropic difference is invisible to the harness — which is exactly the property hango had to build by hand.
• How to debug this layer: provider.resolve spans show which credential source a run used; hango.provider.resolutions by source and error.type catches misconfiguration; because both adapters emit identical chat spans, the Grafana and Langfuse views compare providers side by side on latency, tokens and error rate.
• Exercises: point the OpenAI adapter at a local Ollama base URL and run the same agent; implement a stub CredentialLookup in a test and watch the registry resolve through it.
• Time budget: 8h implementation + 2h learning doc.

Success Criteria

• go build ./... && go vet ./... && go test ./... pass.
• Golden tests cover text, single tool call, two parallel tool calls with interleaved fragments, late-arriving tool name, length finish, and context-length error.
• parity_test.go shows the same normalized event sequence from both adapters for an equivalent scripted conversation, and the phase-5 loop suite passes against both with no loop changes.
• Attribute-parity test: both adapters emit the same chat span attribute keys.
• CredentialLookup is the only path to a credential; a registry test with a stub lookup proves the registry never reads config or store directly.
• Startup rejects a non-https, non-loopback HANGO_OPENAI_BASE_URL with a config error naming the variable.
• Saving an agent with an unresolvable provider or thinking on an incapable provider returns 422 (tests).
• With a provider reporting PromptCaching: false, no cache_control appears on any block of the outgoing request (test).
• In Grafana, runs on both providers appear in the same chat span views and token/latency metrics split cleanly by gen_ai.provider.name.

Risk Assessment

Rollback

No migration to revert. Removing the OpenAI factory from the registry returns the system to Anthropic-only behaviour without touching agents whose provider is anthropic; agents pointing at openai fail validation with a clear 422 rather than failing mid-run.