Agents, sessions, messages

Overview

Add the three tenant-owned domain tables the harness runs on: agents (a user-authored configuration: model, system prompt, tool allowlist, JSON config), sessions (a conversation thread belonging to an agent), and messages (append-only rows with jsonb blocks, one row per message, addressed by (session_id, seq)). Add CRUD APIs for all three and the session-key convention. Still no LLM call: this phase is the persistence substrate the loop writes into.

Requirements

Functional

• agents: created by a user inside their tenant; fields name, slug, model, provider, system_prompt, tools jsonb (allowlist), config jsonb, created_by.
• sessions: belong to an agent, carry a human-readable key, a title, a status, created_by, and counters used later by compaction. created_by exists so intra-tenant visibility can be enforced on sessions and, transitively, on their messages and runs.
• messages: append-only; (session_id, seq) unique; role, blocks jsonb, provider_message_id, stop_reason, usage jsonb, created_at. No updates, no deletes outside session deletion.
• API: POST/GET/PATCH/DELETE /v1/agents, POST/GET/DELETE /v1/sessions, GET /v1/sessions/{id}/messages (paginated by seq), POST /v1/sessions/{id}/messages for appending a user message directly.
• Appending a message allocates seq atomically under concurrency (no gaps required, no duplicates allowed).
• Session key convention: agent:{agent_slug}:{source}:{external_id}; GET /v1/sessions?key= resolves or the create endpoint upserts by key.
• Per-route authorization matrix for this phase, added to `internal/server/authz.go`. "own" means `created_by = principal.UserID`.

• Denials are 404, not 403, on read routes, so a `member` cannot probe for the existence of another user's agent or session. The scoping predicate (`created_by = $n` for members) lives in the store methods, not in the handlers, so a new handler cannot forget it.

Non-functional

• Cross-tenant and intra-tenant cross-user isolation cases added for agents, sessions, messages (a member cannot read or delete another user’s agent or session; owner/admin can).
• provider.Block persists and reloads byte-stable through jsonb, including thinking signatures.
• Message listing is index-backed: (session_id, seq).
• Deleting an agent cascades to its sessions and messages inside one transaction.

Architecture

// internal/store/types.go (additions)
type Agent struct {
	ID, TenantID, CreatedBy uuid.UUID
	Name, Slug, Model, Provider, SystemPrompt string
	Tools  []string
	Config json.RawMessage
	CreatedAt, UpdatedAt time.Time
}

type Session struct {
	ID, TenantID, AgentID, CreatedBy uuid.UUID // Red team: D2
	Key, Title, Status string
	LastSeq int
	CreatedAt, UpdatedAt time.Time
}

type Message struct {
	ID, TenantID, SessionID uuid.UUID
	Seq        int
	Role       provider.Role
	Blocks     []provider.Block
	StopReason string
	Usage      json.RawMessage
	CreatedAt  time.Time
}

// internal/store/store.go (Scoped additions)
type Scoped interface {
	// ...
	Agents() AgentStore
	Sessions() SessionStore
	Messages() MessageStore
	InTx(ctx context.Context, fn func(Scoped) error) error // scope preserved inside the tx
}

type MessageStore interface {
	Append(ctx context.Context, sessionID uuid.UUID, msgs ...Message) ([]Message, error) // allocates seq
	List(ctx context.Context, sessionID uuid.UUID, afterSeq, limit int) ([]Message, error)
	History(ctx context.Context, sessionID uuid.UUID) ([]provider.Message, error)        // loop-ready
}

-- migrations/00002_agents_sessions_messages.sql
create table agents (
  id uuid primary key,
  tenant_id uuid not null references tenants(id) on delete cascade,
  created_by uuid not null references users(id),
  name text not null,
  slug text not null,
  provider text not null default 'anthropic',
  model text not null,
  system_prompt text not null default '',
  tools jsonb not null default '[]'::jsonb,
  config jsonb not null default '{}'::jsonb,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now(),
  unique (tenant_id, slug)
);
create index agents_tenant_created_idx on agents (tenant_id, created_at desc);

create table sessions (
  id uuid primary key,
  tenant_id uuid not null references tenants(id) on delete cascade,
  agent_id uuid not null references agents(id) on delete cascade,
  created_by uuid not null references users(id), -- Red team: D2 — intra-tenant visibility
  key text not null,
  title text not null default '',
  status text not null default 'active',
  last_seq int not null default 0,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now(),
  unique (tenant_id, key)
);
create index sessions_tenant_agent_idx on sessions (tenant_id, agent_id, updated_at desc);
create index sessions_tenant_creator_idx on sessions (tenant_id, created_by, updated_at desc); -- Red team: D2

create table messages (
  id uuid primary key,
  tenant_id uuid not null references tenants(id) on delete cascade,
  session_id uuid not null references sessions(id) on delete cascade,
  seq int not null,
  role text not null,
  blocks jsonb not null,
  stop_reason text not null default '',
  usage jsonb,
  created_at timestamptz not null default now(),
  unique (session_id, seq)
);
create index messages_tenant_session_seq_idx on messages (tenant_id, session_id, seq);

Instrumentation

• Spans: each store method runs inside the caller’s span and produces pgx CLIENT child spans automatically (phase 2 tracer). MessageStore.Append and InTx add explicit INTERNAL spans (store.messages.append, store.tx) with hango.session.id, hango.message.count and, for InTx, the statement count, because sequence allocation under row lock is the one place contention will show up.
• Attributes: handlers set hango.agent.id and hango.session.id on the request span as soon as they are resolved; internal/telemetry.ContextLogger is extended so the same ids appear on every log line.
• Metrics: hango.messages.appended counter (attributes role), hango.sessions.active gauge, hango.message.blocks histogram (blocks per message).
• Log fields: every handler logs one completion line with hango.agent.id, hango.session.id, seq range, latency.

Related Code Files

Create

• migrations/00002_agents_sessions_messages.sql.
• internal/store/pg/agent.go, internal/store/pg/session.go, internal/store/pg/message.go, internal/store/pg/tx.go.
• internal/store/sessionkey.go — BuildKey(agentSlug, source, externalID) and ParseKey.
• internal/server/handlers_agents.go, internal/server/handlers_sessions.go, internal/server/handlers_messages.go.
• internal/server/dto.go — request/response structs and validation (no domain types on the wire).
• internal/store/pg/agent_test.go, session_test.go, message_test.go (integration).
• docs/learning/04-sessions-and-messages.md.

Modify

• internal/store/store.go, internal/store/types.go, internal/store/pg/scoped.go — new sub-stores and InTx.
• internal/store/pg/tenancy_test.go — add agents, sessions, messages cases.
• internal/server/server.go — mount the new routes on the tenant-authed group.
• docs/system-architecture.md, docs/learning/README.md.

Implementation Steps

1. Write migrations/00002_agents_sessions_messages.sql with the DDL above and matching -- +goose Down drops in reverse order.
2. Extend internal/store/types.go with Agent, Session, Message. Message.Blocks is []provider.Block — the store package may import provider (one-way; provider never imports store).
3. Implement InTx in internal/store/pg/tx.go: begin a pgx.Tx, build a new scoped bound to the same Scope over the tx querier, run the callback, commit or rollback. This is the only way later phases get multi-statement atomicity without losing scoping.
4. Implement AgentStore CRUD with pgx.CollectRows + RowToStructByName. Tools and Config bind as []byte/json.RawMessage.
5. Implement SessionStore: Create (sets created_by from the scope; ON CONFLICT (tenant_id, key) DO NOTHING + re-select for upsert-by-key), Get, ListByAgent, Delete. Every read and mutation method appends AND created_by = $n when Scope.Role == member; the predicate is built in one helper (visibility(scope)) shared by the agent, session, message and later run stores, so there is one place to audit. visibility(scope) returns an error for any role other than owner, admin or member, including the empty role, so a scope built without a role reads nothing rather than the whole tenant.
6. Implement MessageStore.Append: inside InTx, UPDATE sessions SET last_seq = last_seq + $n ... RETURNING last_seq to reserve a contiguous block of sequence numbers under row lock, then a multi-row INSERT. This makes concurrent appends safe without an advisory lock. Add a concurrency test running 20 parallel appends and asserting 20 unique consecutive seq values.
7. Implement MessageStore.History: select ordered blocks and rebuild []provider.Message for the loop; assert in a test that a thinking block survives the round trip with its signature.
8. Write internal/store/sessionkey.go and its unit test, including rejection of keys containing : in the external id.
9. Write the DTO layer and handlers. Validation: agent slug pattern, model non-empty, tools an array of known-format names (the registry does not exist until phase 5, so only shape is validated here). All handlers derive the store from store.For(c.Request().Context()).
10. Call `SetAuditAction` in every mutating handler of this phase: `agent.create`, `agent.update`, `agent.delete`, `session.create`, `session.delete`. Appending a message declares no action of its own: the append-only `messages` rows are already the record, and duplicating them in `audit_events` would double the write volume of the hottest path for no new information.
11. Add the three new cases to tenancy_test.go, each with both the cross-tenant and the intra-tenant cross-user assertion, and add the route rows to authz_test.go.
12. Update docs/system-architecture.md with the data model section and write the learning doc.

Learning Doc Outline

docs/learning/04-sessions-and-messages.md must contain:

• Concept: conversation state as an append-only log, and why per-message addressing matters before compaction and subagents exist.
• Design choices: one row per message, jsonb blocks, (session_id, seq) uniqueness, last_seq reservation under row lock, session-key convention, agent as data (not code).
• Trade-offs: append-only rows vs one jsonb blob per session (goclaw’s choice) — write amplification, per-turn deltas, checkpoint growth (deepagents’ DeltaChannel needed to cut O(N^2) growth), and why append-only wins for a server.
• goclaw comparison: SessionData as a []providers.Message blob plus summary and token counters; the structured agent:{agentKey}:{channel}:{direct|group}:{id} session key and why it uses agentKey not UUID.
• How to debug this layer: to answer “why does this session look wrong”, filter Loki by hango.session.id to get the full append history, then open the store.messages.append spans in Tempo to see sequence allocation order and any contention; a slow history load shows up as a long pgx span under the request span.
• deepagents comparison: messages live in LangGraph state with a checkpointer; thread_id is the session id; DeltaChannel exists exactly because full-transcript re-persistence is O(N^2).
• Why hango diverges: durable rows are a prerequisite for resumable runs and HITL.
• Why the creator predicate belongs in the store's `visibility(scope)` helper and not in handlers.
• Exercises: add a message, inspect the jsonb in psql, break seq allocation on purpose and see the unique violation.
• Time budget: 10h implementation + 2h learning doc.

Success Criteria

• go build ./... && go vet ./... && go test ./... pass.
• go test ./internal/store/pg/ -run TestCrossTenant -v now shows cases for agents, sessions, messages.
• A member gets 404 for another user’s agent, session and message list, and a 404 (not 403) on delete; owner and admin get 200 for the same resources (tests).
• visibility(scope) with an empty or unknown role returns an error and the store method returns no rows (unit test).
• Parallel-append test: 20 goroutines append to one session and produce seq 1..20 with no duplicates.
• A message containing a thinking block with a signature reloads byte-identical through History.
• DELETE /v1/agents/{id} removes the agent, its sessions and its messages; a test asserts zero rows remain.
• Creating, updating and deleting an agent and creating and deleting a session produce audit_events rows with actions agent.create, agent.update, agent.delete, session.create, session.delete, each with the right target_id (integration test).
• GET /v1/sessions/{id}/messages?after_seq=0&limit=50 returns ordered messages with a next cursor.
• In Grafana, a session request trace shows store.messages.append and pgx spans, and hango.agent.id/hango.session.id are present on both the spans and the correlated Loki lines.

Risk Assessment

Rollback

goose down one step drops messages, sessions, agents. The identity spine and server from phases 1-2 keep working; the new route group is registered in one place in server.go.