Praesidium2 v1.12

Service Responsibility Distribution

Each axis represents a system capability. The radar shows how much each service is involved in that capability (0 = not involved, 10 = primary owner). Notice how the three services have almost zero overlap — clean separation of concerns.

Service Topology

Environment Comparison

Full local development design including Tiltfile, developer workflow, Ollama model recommendations, and self-hosted deployment guide: docs/superpowers/specs/2026-04-16-local-development-design.md

CI/CD Pipeline

Pipeline: Faber CI (.faber.yml per repo) → GHCR → ArgoCD → EKS. Rollback via git revert on values file.

Centralized logging across all 5 services. Debugging distributed issues is impractical without it.

{
  "timestamp": "2026-04-02T14:30:00Z",
  "level": "INFO",
  "service": "agent-worker",
  "correlation_id": "uuid",
  "conversation_id": "uuid",
  "job_execution_id": "uuid",
  "agent_id": "uuid",
  "message": "Task execution completed",
  "duration_ms": 1234
}

Correlation IDs: UUID generated at gateway, propagated via HTTP headers (X-Correlation-ID), Celery task kwargs, Redis pub/sub messages, and qqq API calls. All log entries include correlation_id for end-to-end request tracing.

OpenTelemetry SDK integrated into agent-gateway (FastAPI) and agent-worker (Celery) from Day 1.

Authentik + qqq Pattern

Same proven pattern as me-health-portal. Authentik is the identity provider (IdP), qqq enforces all permissions server-side. No permission logic lives in application code — it is all externalized into Authentik groups and qqq RecordSecurityLock.

Authentik Group Structure

IAM-Style Policy Format

Permissions are expressed as JSON policy documents attached to Authentik role groups. When a user belongs to multiple role groups, their policies are merged (union of all Allow statements, any Deny overrides).

{
  "PolicySpec": "praesidium-iam",
  "PolicyVersion": "1.0.0",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["read", "insert", "edit"],
      "Resource": ["project", "task", "conversation"]
    },
    {
      "Effect": "Deny",
      "Action": ["delete"],
      "Resource": ["agent_template"]
    }
  ]
}

Row-Level Security

All entity tables get RecordSecurityLock on instance_id, enforced by qqq at query time. Same pattern as me-health-portal's mhCompanyId.

Agent-workers authenticate to the qqq API using a scoped service account JWT issued by Authentik. This is NOT an all-access admin account.

Scoped IAM Policy

{
  "PolicySpec": "praesidium-iam",
  "PolicyVersion": "1.0.0",
  "Statement": [
    { "Effect": "Allow", "Action": ["read"], "Resource": ["*"] },
    { "Effect": "Allow", "Action": ["insert", "edit"],
      "Resource": ["task", "task_comment", "message", "conversation",
                    "conversation_participant", "activity_log", "memory",
                    "job_execution", "escalation", "artifact",
                    "human_interaction", "agent_skill", "knowledge_chunk"] },
    { "Effect": "Allow", "Action": ["edit"],
      "Resource": ["agent", "scheduled_job", "conversation"] },
    { "Effect": "Deny", "Action": ["delete"],
      "Resource": ["project", "agent_template", "team_member", "integration_config"] }
  ]
}

Workers can read everything within the instance, write to operational tables (tasks, messages, memories, activity logs), and update agent execution status. Workers CANNOT delete projects, agent templates, team members, or integration configs.

Service Account JWT issued by Authentik, scoped to the Agent's Instance ID. Worker presents this JWT to qqq for all hydration and CRUD operations. No direct database access — all data flows through authenticated qqq REST endpoints.

Execution Patterns

Memory System Flow

On next hydration: semantic search with task/conversation context, recency bias + importance filter, inject top-K memories into system prompt.

All state changes from the agent runtime go through qqq REST APIs. These are the API call patterns for each execution type:

Task-type-specific queues with tuned visibility timeouts to prevent duplicate execution of long-running tasks:

Workers subscribe to all three queues. Routing is determined at dispatch time by the gateway based on task type.

To reduce HTTP round-trip overhead (a single task execution generates 10-20 REST calls), qqq exposes composite endpoints using its include query parameter. No custom API code needed — qqq resolves joins from ERD relationships.

To minimize latency and prevent logic drift, the Worker hydrates 100% of its state through a single composite endpoint. This replaces the multi-call hydration flow described above with an atomic, internally-scoped API call.

AGENT_MANIFEST Response Structure

{
  "agent": {
    "id": "uuid",
    "name": "Forge Agent",
    "persona": "Senior DevOps Engineer...",
    "instance_id": "uuid-123"
  },
  "config": {
    "primary_model": {
      "name": "claude-3-5-sonnet",
      "provider_url": "https://api.anthropic.com",
      "context_window": 128000
    },
    "fallback_model": { "name": "gpt-4o" }
  },
  "secrets": [
    { "key": "GITHUB_TOKEN", "arn": "arn:aws:secretsmanager:us-east-1:..." },
    { "key": "JIRA_API_KEY", "arn": "arn:aws:secretsmanager:us-east-1:..." }
  ],
  "tools": [
    { "id": "uuid", "name": "git_commit", "risk_level": "CRITICAL" }
  ]
}

Key design choice: Secrets in the manifest are ARNs only — never plain-text values. The Worker pod fetches actual secrets directly from AWS Secrets Manager using IRSA (see Blind Secret Pipeline below).

Security Mandate: Plain-text secrets MUST NEVER be stored in the Praesidium database or passed through the Agent Gateway. The database stores only AWS Secret ARNs.

Compliance: This architecture satisfies SOC2 CC6.1 (logical access), HIPAA §164.312(a)(1) (access controls), and ensures no credentials leak through application logs, database backups, or API responses.

The denormalized counters on agent (total_tasks_completed, total_tokens_used, total_cost_usd) are CACHE values — updated on write for fast reads but can drift. A periodic reconciliation job ensures correctness.

Counters are treated as cached aggregates — source tables (task, message, activity_log) are authoritative. Agent detail page can also compute live values on demand.

Unified search across all Praesidium entities, powered by pgvector semantic embeddings. This endpoint drives the frontend Command Palette (Cmd+K).

GET /api/search?q={query}&scope={documents|agents|tasks|all}

{ "results": [
    { "type": "task",
      "id": "uuid",
      "title": "...",
      "relevance_score": 0.92,
      "snippet": "..." }
  ],
  "total": 42,
  "query_ms": 23 }

Instance-scoped via JWT praesidiumInstanceId. Results respect IAM policies — users only see entities they have read permission for.

Scaling Strategy

Failure Scenarios & Recovery

Agent personality (WWE, LOTR, Grey's Anatomy, Star Wars themes) is configurable per team instance. Each business unit chooses their own personality theme — or uses professional mode for external-facing contexts. Personality does not affect agent capability, only communication style.

Design principle: Themed personalities are opt-in. The personality_traits JSONB field on agent_template stores personality configuration. When an agent interacts with external-facing tools (e.g., sending emails to clients, posting to public channels), the personality layer is automatically bypassed in favor of professional tone. The system prompt dynamically adjusts based on the tool's context_mode setting.

Full Tech Stack

Dual API Architecture

Two API targets — qqq for data, Agent Gateway for AI interactions. Same Authentik JWT for both. Distinct root prefixes make each service immediately identifiable in logs, network traces, and dev tools; versioning allows non-breaking evolution of either API independently.

10 Primary Pages

Standard qqq CRUD endpoints for the scheduled_job table, plus custom actions for on-demand execution and history retrieval.

Celery Beat Integration: When a user creates, edits, or deletes a schedule via these endpoints, Celery Beat picks up the change on its next heartbeat cycle (every 30s). The next_run_at column is recomputed from cron_schedule after each save. This ties directly into the G-002 scheduler_heartbeat HA mechanism — only the elected leader node processes schedules.

What is the Tool Forge?

The Tool Forge is where you work one-on-one with an AI agent to build custom tools that any agent on your team can use. You describe what you need in plain English, and the forge agent writes the code, tests it, and publishes it — no programming knowledge required.

6-Step Forge Workflow

Powered by LangGraph state graph with checkpointing. Human-in-the-loop at review stage. Git-backed versioning with immutable snapshots.

Forge Workflow — Step Details

Each step is a LangGraph node with durable checkpointing. The graph pauses at human-in-the-loop boundaries.

Tool Execution Model — Where Each Type Runs

Custom Tool Pod Lifecycle

Warm Pod Pool Optimization

For frequently-used CUSTOM tools, maintain a pool of warm containers pre-loaded with tool code. Reduces cold-start latency from ~5s (k8s Job creation) to <200ms (HTTP call to warm pod).

Tool Interface Contract

Every tool — regardless of category — conforms to the same contract stored in the tool table.

The CMO (Bryan Chupp) wants to build an Amazon keyword research tool similar to Helium 10's Cerebro. She describes what she needs to the Forge agent via chat, and together they build, test, and publish it.

Building the Suite

With cerebro_keyword_lookup published, Bryan and the marketing team forge 4 more tools using the same workflow — creating a complete Amazon PPC toolkit.

Seed Models (P0)

Capability Tiers

Model Assignment Flow

Models cascade from template defaults through agent overrides to job-specific overrides.

LLMClient Protocol Abstraction

A single Python Protocol interface — V1 implements AnthropicClient only. The LLMClient Protocol enables additional providers post-V1 without changing agent runtime code.

Provider Fallback Flow Post-V1

When a provider goes down, agents automatically fall back to configured secondary models. V1 uses Anthropic only; fallback activates when additional providers are configured post-V1.

Secret Scopes

secret_ref Table Schema

UNIQUE: (scope_type_id, project_id, tool_id, agent_id, name) — one secret per name per scope

Resolution Flow (Narrowest Scope Wins)

Agent-scoped secrets override all others when the specific agent is executing.

Storage Model

Integration Registry Pattern

Every external system is represented by a connector class implementing a uniform interface. On startup, agent workers load all registered connectors from integration_config where is_active = true. Each connector's tool_factory() produces INTEGRATION category tools.

class IntegrationConnector:
    async def health_check() -> HealthResult
    async def authenticate(config: IntegrationConfig) -> AuthSession
    async def refresh_token(session: AuthSession) -> AuthSession
    def tool_factory() -> list[Tool]

Credential Management Layers

Webhook Receiver Endpoint

Shared endpoint for inbound callbacks from all integrations:

Error Handling: Circuit Breaker Pattern

Cross-Integration Coordination

Multi-integration workflows are orchestrated via tasks/events, not connector-to-connector calls.

Full catalog across 4 team types. Each follows the same IntegrationConnector pattern.

Agent Performance Metrics

Agent performance is derived from existing tables, not a separate metrics store. Aggregated API endpoints computed on read.

System Health Monitoring

Meeting Analytics

All meeting data flows through the conversation model. A meeting is a conversation where type_id resolves to STANDUP, HUDDLE, or ESCALATION, linked to a scheduled_meeting record.

Token cost per meeting: conversation.total_cost_usd. Per-participant breakdown via SUM(message.cost_usd) grouped by message.author_id.

Infrastructure Metrics (12 Prometheus Metrics)

Infrastructure layer: EKS = CloudWatch Container Insights (pod CPU/memory, ALB, RDS, ElastiCache). Local = Prometheus scraping + Grafana dashboards.

Cost Management

Token Tracking

Every LLM interaction records token usage at the message level. Costs computed using Anthropic pricing table maintained in-memory at the gateway.

Cost Aggregation Views

Implemented as qqq API endpoints backed by SQL aggregations (not materialized views).

Budget Controls

Cost Optimization Strategies

Build Team & Approach

The full engineering leadership team is listed below. The Praesidium2 build team is James Maes (CTO) + 2 engineers + unlimited Claude Code agents. Continuous deployment, daily standup, weekly demo. 7-phase build plan: Pre-Build KPI baseline, 6 build phases, and 1 stabilization phase. Phases can be completed faster than calendar weeks when AI agents are productive. Engineers provide the quality review gate that ensures production-grade output.

Phases are not calendar weeks. The team may complete phases faster than estimated — the goal is to move through phases as quickly as quality allows.

7-Phase Build Plan

Re-sequenced per the “Steel Thread” strategy: Tool Runner (Hands) before Agent Workers (Brain) to prevent testing deadlocks. Building the execution sandbox first means every subsequent component can be integration-tested against real tool execution from Day 1.

3 engineers (CTO + 2 devs) + unlimited Claude Code AI agents accelerated in parallel. Existing AWS EKS + Faber CI/CD + ArgoCD. Engineers provide the quality review gate that ensures production-grade output.

After Business Review scope analysis and Technical Audit remediation, the committed timeline is 7 phases (Pre-Build + 6 build phases + 1 stabilization phase) with scope cuts from findings B1 (Template Library deferred), B2 (Tool Forge deferred), B8 (Memory System deferred), and B14 (Knowledge System deferred to V1.5). Pre-Build KPI baseline phase runs before the build starts.

The 7-phase timeline is the commitment used for planning, staffing, and stakeholder communication. A Pre-Build KPI baseline phase runs before the build starts. See Rollout Strategy tab for team-by-team deployment phases.

Phase-by-Phase Breakdown

Key insight: Tool Runner (Phase 2) is built before Agent Workers (Phase 3). This “Hands before Brain” sequencing ensures agents can execute real tools from the moment they come online, preventing “brain with no hands” integration testing deadlocks.