Skip to content

Bring your own formation model

By default, memory formation runs on the model Jennah provides, and its cost is counted in formation units. An organization on the Pro, Scale, Team, or Enterprise plan can run formation on its own model instead, in one of two ways:

  • Amazon Bedrock (for example Claude Sonnet 5.5) in your AWS account.
  • The Claude API, with an API key from your Anthropic organization.

With a formation model set, every formation for your organization runs on that model, across every location, and consumes no formation units: you pay AWS or Anthropic for the inference.

Only an interactively authenticated root or admin user can configure a custom formation model; API keys cannot modify this setting.

How Jennah reaches your model

Bedrock: No static AWS credentials are shared with Jennah. Instead, Jennah uses AWS STS to assume an IAM role in your AWS account. The role's trust policy requires an organization-specific external ID to prevent confused-deputy access.

Claude API: You provide an Anthropic API key, which Jennah uses for all formation calls in your organization. The key is write-only: it is stored encrypted, never returned in full by read APIs (which expose only the last four characters), and deleted when you remove the model, switch to Bedrock, rotate the key, or delete the organization.

Setting up Bedrock

  1. Retrieve the trust policy parameters for your organization:

    jnh formation-model prepare --provider bedrock
    

    This prints the broker principal ARN to trust, the external ID issued to your organization, and the STS session name. Calling it again prints the same values.

  2. Create the IAM role. Use the Jennah CloudFormation template: prepare also prints a Launch stack link (the console's Formation model settings have the same link as a button). Following it opens the AWS CloudFormation console with the template, your external ID, and the broker principal already filled in. Review the stack, check the IAM acknowledgment, and create it. The stack's RoleArn output is the role you register in step 4.

    The stack creates a role that trusts only the broker principal, and only with your external ID. The role is granted only bedrock:InvokeModel on the model named by the ModelId parameter (default anthropic.claude-sonnet-5-5), invoked directly or through any inference profile for it. Delete the stack to revoke access. While the model is still set, formations then fail at the access step and never fall back to Jennah's model; remove the model to return to Jennah's model, or recreate the stack with the same external ID.

    To create the role manually instead, use the following trust policy:

    {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Principal": {"AWS": "<principal from prepare>"},
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {"sts:ExternalId": "<external ID>"}
        }
      }]
    }
    
  3. If you created the role manually, attach an IAM policy that allows bedrock:InvokeModel on the foundation model and on any inference profile for it. The CloudFormation template includes this already:

    {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Action": "bedrock:InvokeModel",
        "Resource": [
          "arn:aws:bedrock:*::foundation-model/<model>",
          "arn:aws:bedrock:*:<account>:inference-profile/<profile>"
        ]
      }]
    }
    

    For Claude Sonnet 5.5, <model> is anthropic.claude-sonnet-5-5 and <profile> is global.anthropic.claude-sonnet-5-5. Enable the model in the Bedrock console for your account, in the region you will name.

  4. Set the model:

    jnh formation-model set --provider bedrock \
      --role-arn arn:aws:iam::<account>:role/<role> \
      --region ap-northeast-1 \
      --model global.anthropic.claude-sonnet-5-5
    

    Some models are offered only through an inference profile (with a global. or regional prefix) rather than a bare model ID. AWS lists the inference profile IDs, with the models and regions each one covers, in Supported Regions and models for inference profiles. The CloudFormation template's ModelId parameter takes the model ID without the prefix (anthropic.claude-sonnet-5-5); --model takes the profile ID (global.anthropic.claude-sonnet-5-5).

Setting up the Claude API

  1. Create an API key in the Claude Console. Use a dedicated workspace with its own spend limit, so that formation spend is isolated on your bill and capped if the key is exposed. The key's workspace must be allowed to use the model you choose.

  2. Choose a model, for example claude-sonnet-5-5, and optionally an inference geography. With --inference-geo us, the Claude API runs every formation's inference in the United States; without it, the API's default applies, which may run inference anywhere it serves.

  3. Configure the model. set reads the API key from standard input when piped, or prompts for it interactively without echoing (the key cannot be passed via command-line flags):

    jnh formation-model set --provider anthropic \
      --model claude-sonnet-5-5 --inference-geo us \
      < anthropic-key.txt
    

    The key is never shown again. jnh formation-model get shows it as key ...abcd, the last four characters.

To change the settings and keep the key, run set --provider anthropic again without a key: with a Claude API model already set, the stored key is kept. Changing from Bedrock to the Claude API always needs a key.

To rotate the key, create a new one, run set with it, then revoke the old one in the Claude Console. set checks the new key before saving it, so formations keep using the old key until the new one is in place, and the old key is destroyed as soon as it is replaced.

Jennah calls the configured Claude model directly and does not fall back to another model if a request is declined.

Setting checks the model first

Before saving changes, set executes a validation formation call using the target model and configuration. If validation fails, the existing configuration remains unchanged and the error identifies the failing step:

Step Meaning Usual fix
access The role could not be assumed, or the API key was rejected Check the trust policy and external ID, or that the key is active and its workspace may use the model
model The model is not enabled, not found, or refused a setting Enable the model in that region, check the ID, or lower --max-output-tokens
answer The model answered, but not with usable structured output Choose a model that supports structured output

The check costs your provider one model call.

To run the same check later without changing anything, use jnh formation-model check or Check now in the console's Formation model settings. It reports whether the model works or which step failed.

Settings

Flag Default Meaning
--thinking provider default low, medium, or high: how much the model reasons before answering
--max-output-tokens 32,000 Bound on one call's output tokens, reasoning included (up to 65,536)

Every formation makes two model calls (extraction, then reconciliation), and reasoning is billed by your provider as output. On Claude Sonnet 5.5, the provider default thinking level is high.

On Claude, medium occasionally returns an empty extraction. Jennah reports that as a failed formation you can retry, rather than as a formation that found nothing.

No fallback

With a formation model set, formation never runs on Jennah's model, even when yours cannot be reached. A formation that cannot reach your model fails, writes nothing, and names your model in the error:

Situation Status
Role cannot be assumed, or API key rejected FAILED_PRECONDITION (identifies the access step)
Model not found, not enabled, or refused parameters FAILED_PRECONDITION (identifies the model and region)
Your Claude model declined the request FAILED_PRECONDITION (identifies the model and the refusal)
Provider throttled or failed after retry UNAVAILABLE (identifies the model)
Call took too long DEADLINE_EXCEEDED (as on Jennah's model)
Call reached its output bound ABORTED (as on Jennah's model)
Your plan no longer includes a formation model FAILED_PRECONDITION (remove the model or upgrade)

Removing the formation model is the only way back to Jennah's model:

jnh formation-model remove

Billing

  • A formation on your model consumes 0 formation units, and its receipt reports formationUnits: 0. It does not count toward your allowance or overage.
  • Its token usage is still recorded. In token usage grouped by model, it appears under a name like bedrock:ap-northeast-1/global.anthropic.claude-sonnet-5-5 or anthropic:us/claude-sonnet-5-5, so you can reconcile it against your provider's bill.
  • The billing page shows how many formations were sent to your model this period, including any that failed there.
  • Formations on your model still count toward your plan's limit on formations in progress, and the submission size limit still applies.

Where inference runs

Formation inference runs wherever your model runs. A Bedrock inference profile with the global. prefix may run inference outside the region you name; a regional model or profile keeps it in that region. On the Claude API, the inference geography you set decides it. Choose accordingly if your conversations must stay in one region.

Quality

Jennah's formation is measured and tuned on the model Jennah provides. Another model may extract and reconcile differently: for example, it may combine several facts about one person into one fact. Check formed memory on a sample of your own conversations before relying on it.

Reading and auditing

jnh formation-model get

get shows the active model, its settings, who set it and when, and the setup values issued to your organization. It never shows a credential: Bedrock stores none, and a Claude API key is shown only as its last four characters. Every set and remove is recorded with the administrator who made it, and for the Claude API with the key's last four characters, never the key.