Skip to content

Agents

Every agent, skill, rule and runner is factory config: a file in a factory repo with an owner, versioned by git, which the API can read but never change. An agent's scopes list what it may touch, and the API enforces them on every call, whatever the prompt says. Workflows use agents in two step types:

Factory config

flowchart LR
    R1[Rule · org] -.applies to every agent.-> A
    R2[Rule · team / repo] -.applies by owner or repo.-> A
    S[Skill] -- listed on --> A[Agent<br/>owner · version · scopes]
    A -- used by --> ST1[agent step]
    A -- used by --> ST2[code step]
    E[Environment] -- used by --> ST2
    M[Memory lessons] -.injected by subject / repo.-> ST1 & ST2

All of them are defined in factory repos (agents/, skills/, rules/, runners/, plus webhooks/ for webhook sources; see examples/demo), and merging a change applies it. The API stores them in tables of their own, in the Postgres schema config (agents, skills, rules, runners, webhook_sources). They aren't catalog entities: read them with GET /v1/agents, /v1/skills, /v1/rules and /v1/environments, or a factory's files. Memory lessons are injected into the same steps, but they're runtime data (see Memory).

Versions are commits. An agent's, skill's, rule's or runner's version is the commit that last changed it (the first 12 characters of the sha), set when its factory is applied. Files never say version:: a file that does fails the sync. Every run records the versions it used, so what an agent was told is one git show <version>:agents/<id>.yaml away. The Factory page's Agents tab links each version to its commit.

Agents and scopes

kind: Agent
identifier: triager
title: Triager
owner: sre                                # required
model: claude-opus-5-5                    # default FACTORYAL_DEFAULT_MODEL
effort: high
scopes: [catalog:read, "catalog:write:issue"]
skills: [severity-guide]
system_prompt: You triage software issues...
memory_review: auto                       # always: every lesson waits for review (see Memory)
Scope Allows
catalog:read read blueprints and entities, search
catalog:write write entities of any user blueprint
catalog:write:<blueprint> write entities of that blueprint only
repo:write push branches to any repository in the catalog (coding agents)
repo:write:<repo> push branches to that repository only
memory:write / memory:write:<subject> record lessons (the remember tool) about any / one subject
proposals:write read the files of its own factory's resources and propose changes to them as PRs (the read_factory_file and propose_change tools, see Learning)

Agents can never get admin. Factory config can't be written through the API at all, and other platform-managed blueprints (names starting with _, such as _memory) are admin-only, so an agent can't widen its own scopes, edit its rules or approve its own lessons. The tools offered to an agent are filtered by its scopes, but the API is what enforces them.

Action API
Publish commit to the factory repo; the API can't write them
List GET /v1/agents, /v1/skills, /v1/rules, /v1/environments

Skills

Reusable instructions, listed on agents by identifier, each with its own version (the commit that last changed it).

kind: Skill
identifier: severity-guide
title: Severity guide
owner: sre
instructions: |
  critical: outage or data loss on a tier-1 service. ...

Rules

Rules are standing instructions that agents follow without listing them. org rules apply to every agent, team rules to agents owned by target, and repo rules to coding steps on the repo target.

kind: Rule
identifier: no-secrets
title: Never commit secrets
owner: platform
scope: org                 # or team / repo, with target: <team or repo>
instructions: Never commit secrets, tokens or .env files.

Config report

GET /v1/config/report lists agents, skills and rules, and flags sprawl and drift in issues: no_owner, duplicate_skill (identical instructions), unused_skill, version_drift (variants of one skill name in use at different commits), rule_drift (recent runs followed a rule as it was before its last change) and benchmark_regression.

Memory

Lessons are catalog data, not factory config: agents write them while they run, so they live in the API, not in a factory repo. They're stored as _memory entities, which only admins can write through the entity API, so an agent can't skip review by writing one directly.

An agent with memory:write[:<subject>] gets a remember tool for recording a lesson (e.g. "run make deps before the tests") with a confidence.

  • Lessons at or above FACTORYAL_MEMORY_AUTO_ACCEPT (0.8) become active at once. The rest wait in the review queue (web UI → Learning → Lessons, or POST /v1/memories/<id>/review).
  • An agent with memory_review: always (default auto) never skips the queue: every lesson it records starts proposed, whatever its confidence. Use it for agents that read untrusted input, so a prompt-injected lesson can't reach other runs unreviewed.
  • Each lesson keeps its source run and expires after FACTORYAL_MEMORY_TTL_DAYS (90).
  • Coding steps get the active lessons about their repo. Agent steps get them for memory: <subject>. Both get at most FACTORYAL_MEMORY_PER_STEP (10) lessons, the most recent first.
  • GET /v1/memories?subject=&status= lists them.

Only agent steps write lessons. code steps on Managed Agents have no Factoryal tools, so they read lessons but can't add any. A retro of a coding run can record lessons for it, as proposed memories.

Agent steps

- id: triage
  type: agent
  agent: triager
  max_turns: 15
  memory: billing                # optional: inject active lessons about this subject
  prompt: Triage issue {{ .inputs.issue }} and record severity on the entity.
  output_schema:                 # optional: the final answer is parsed and validated JSON
    severity: string             # shorthand: workflows.md#output-schemas

The agent loop runs in one activity on the llm queue. The API mints a short-lived token limited to the agent's scopes for each step, and every tool call (list_blueprints, search_catalog, get_entity, upsert_entity, remember) goes through the API with that token (sequence). Agent steps aren't retried by default, because they have side effects.

With FACTORYAL_LLM_PROVIDER=claude_code the loop is the Claude Code CLI instead of the API, so agent steps can run on a Claude subscription (Lightsail → Claude on a subscription). The worker runs claude -p with no built-in tools, no settings and a throwaway home directory. The agent's tools come from an MCP server the CLI starts (factoryal.contexts.agents.application.mcp_server). It offers the tools the agent's scopes allow and calls the API with the same scoped token, so the API still enforces the scopes. Model, effort, max_turns, output_schema, skills, rules and memory work the same way, and the run keeps a transcript of the tool calls.

An agent whose scopes offer no tools (e.g. scopes: [], an agent that only reads its memory lessons) answers in one call, as an llm step does. In a workflow with replay: { capture: true } its requests are captured, with either provider, and can be replayed on other models (Efficiency → Replays).

Coding agents

- id: fix
  type: code
  agent: builder               # agent with repo:write or repo:write:<repo>
  repo: "{{ .inputs.repo }}"   # a `repo` entity in the catalog (its `url` points at GitHub)
  task: Add a CODEOWNERS file assigning the payments team.
  rubric: |                    # optional; a default one is generated from the task
    - .github/CODEOWNERS exists and assigns @acme/payments
  budget_usd: 10               # hard cap on the session's spend (default 25)
  max_iterations: 3            # grading passes
  draft: true                  # open the PR as a draft (default)

The agent works on the branch factoryal/<run>-<step>. It's graded against the rubric and keeps iterating until the change meets it, or until max_iterations or the budget runs out. Then the platform checks that the branch has commits and opens the PR as the GitHub App's bot (sequence).

The output includes pr_url, pr_number, branch, summary and outcome. If a worker dies mid-session, Temporal retries the activity and it reattaches to the same session. Cancelling the run interrupts the session, and the run ends once the session has stopped.

Setup: coding agents need the GitHub App and an ANTHROPIC_API_KEY with Managed Agents access on the coding workers.

Where the agent runs

backend: on the step, on the environment, or FACTORYAL_CODING_BACKEND (default managed_agents):

Backend Runs on GitHub token
managed_agents Anthropic Managed Agents: a sandboxed container per session, graded by Anthropic's outcome grader Injected by Anthropic's git proxy at egress, so it never enters the sandbox
claude_code The Claude Code CLI, headless, on Factoryal's coding workers Passed in a per-command HTTP header, never written to disk; the backend pushes, not the agent
codex The Codex CLI, headless, on Factoryal's coding workers Same as claude_code

The CLI backends need an image built with --build-arg CODING_CLIS=true. Each task gets a fresh workspace. The backend clones, creates the branch, runs the environment's setup, then runs the CLI. The agent's environment carries only its own provider key (ANTHROPIC_* / OPENAI_*) and no Factoryal secrets. After each session a grader (a Claude call with the rubric and the diff) decides whether another pass is needed, and the CLI session resumes with its feedback.

A CLI session runs inside the worker task, so it dies with it (a deploy, a scale-in, a crash). Temporal retries the step on another worker, which starts a new CLI session. If the interrupted attempt had already pushed the step's branch, the retry fetches that branch and continues from its last pushed commit instead of recreating it from the base branch, so its push is still a fast-forward. The agent is told an earlier attempt was interrupted, and the grader sees the whole change since the base.

claude_code can also run on a Claude subscription: set CLAUDE_CODE_OAUTH_TOKEN from claude setup-token on the coding workers, and the CLI uses it instead of ANTHROPIC_API_KEY (Lightsail → Claude on a subscription).

Codex doesn't report cost, so Factoryal prices its tokens with FACTORYAL_CODEX_PRICE_PER_MTOK: [input, output, cached input] in USD per million tokens (default [1.25, 10.0, 0.125]). With only two values, cached input costs a tenth of the input price. Codex counts cached input inside its input tokens; Factoryal records the uncached part as input and the cached part as cache reads, the same split Anthropic reports. Its flags (codex exec --json --sandbox workspace-write ...) can be extended with FACTORYAL_CODEX_ARGS. CLI agents run with the coding worker's permissions, so keep coding workers in a separate service with no other secrets (the CDK stack does).

Environments

environment: <id> points a code step (or a benchmark config) at a runner (one of the factory's runners/).

kind: Environment
identifier: py312
title: Python 3.12
owner: platform
backend: claude_code                  # optional
setup: ["uv sync"]                    # run in the repo before the agent starts
allowed_hosts: [pypi.internal.acme]   # reachable besides github.com
package_managers: true                # PyPI, npm, ...
env: { UV_INDEX: "https://pypi.internal.acme" }   # non-secret only

On Managed Agents, each distinct networking setup gets its own managed environment, and the setup commands go at the top of the agent's instructions. CLI backends run the commands themselves, with a 20-minute limit per command. The step output records the environment and its version (commit).

Several repositories

extra_repos: [ledger, docs] mounts more repos next to repo, each with its own single-repo token (the agent needs repo:write on each). The agent uses the same branch everywhere. Factoryal opens a PR in every repo the agent changed, and comments on each PR with links to the others. pull_requests in the output lists them.

Dry runs and fixed starting points

  • mode: dry_run pushes the branch (factoryal-dry/<run>-<step>) but opens no PR. Instead, the output describes the change: commits, files, additions, deletions.
  • base_ref: <sha or tag> branches from that commit instead of the tip of the base branch, so the same task can run again from the same starting point.

Benchmarks are built on these two options.

PR reviews

A submitted review that requests changes (or comments with feedback) on a PR Factoryal opened starts a revision. The same agent, with the same overrides, gets the original task plus the review's body and inline comments. It works on the PR's own branch (continue_branch: true), so the PR gets new commits, and the revision replies on the PR with what it changed (sequence).

  • Revisions of one PR run one at a time, in order. They stop after FACTORYAL_PR_REVISION_LIMIT (default 3), and Factoryal says so on the PR.
  • Approvals, bot reviews and empty comments start nothing.
  • The GitHub App must subscribe to the Pull request review event.

Overrides

agent and code steps can set model, effort and skills (which replaces the agent's skill list) for that step only, without publishing a new agent version. Skill ids can be templates, such as skills: ["pr-review-{{ .inputs.repo }}"] for one skill per repo: a skill that doesn't exist fails the step. Scopes can't be overridden. The step records the overrides next to the agent version. On Managed Agents, each distinct override gets its own managed agent, so mixing overrides doesn't keep re-versioning the base one. routing on the step picks the overrides per run (Model routing).

The session endpoints (POST /v1/runs/{id}/agent-sessions and /code-sessions) take the same overrides and reject any other key with a 422.