AI Should Recommend, Policy Should Decide - Guardrails for Production Messaging Systems

This is Part 2 of the AI-Assisted Message Queue Patterns series. Part 1 introduced the architecture: SQS → consumer → context → AI recommendation → deterministic policy → action. This article focuses on the boundary that makes that architecture practical: what the model may recommend, what policy must control, and what happens when the model is uncertain, unavailable, or wrong.

The model call is the easy part

Calling a model from a Lambda function is not the difficult architectural problem.

The difficult problem starts after the model responds.

Suppose a message-processing application receives this recommendation:

{
  "priority": "HIGH",
  "route": "PROCESSOR_B",
  "retry": true,
  "retry_delay_seconds": 120,
  "confidence": 0.91
}

What should happen next?

A naive implementation might execute the response directly. That creates an uncomfortable amount of authority for a probabilistic component. The model can produce malformed output. It can recommend a destination that does not exist. It can return a retry interval outside the application's operating limits. It can be unavailable entirely.

The safer architecture is:

Message
   ↓
Context
   ↓
AI Recommendation
   ↓
Validate
   ↓
Apply Policy
   ↓
Execute Approved Action

The principle from Part 1 remains:

AI recommends. Policy decides. The messaging platform executes.

Start with an explicit decision contract

Before invoking a model, define what a valid recommendation actually looks like. For example:

{
  "priority": "HIGH | STANDARD",
  "route": "PROCESSOR_A | PROCESSOR_B",
  "retry": true,
  "retry_delay_seconds": 120,
  "confidence": 0.91,
  "reason": "Short explanation"
}

This is intentionally constrained. The model is not being asked "What should I do with this infrastructure?" It is being asked to choose among a bounded set of actions the application already understands.

That distinction reduces the problem from unrestricted generation to constrained decision support.

Guardrail 1: Validate structure before meaning

Never assume that because you asked for JSON, you received usable JSON. A response can be malformed, missing required fields, wrong types, outside an allowed enum, or syntactically valid but operationally impossible.

Validation should happen before the recommendation reaches execution logic.

def validate_decision(decision):
    required = ["priority", "route", "confidence"]

    for field in required:
        if field not in decision:
            return False

    if decision["priority"] not in {"HIGH", "STANDARD"}:
        return False

    if decision["route"] not in ALLOWED_ROUTES:
        return False

    if not 0 <= decision["confidence"] <= 1:
        return False

    return True

If validation fails:

Invalid model response
        ↓
Deterministic fallback

Not:

Invalid model response
        ↓
Best guess

Guardrail 2: Policy before execution

A valid response is not automatically an allowed response. Imagine the model returns:

{
  "route": "PROCESSOR_B",
  "confidence": 0.97
}

High confidence does not grant permission. Policy still needs to answer: Is PROCESSOR_B an approved destination? Is this workload allowed to use it? Does the message satisfy required constraints? Is routing there currently enabled? Does the action violate any workload-specific rule?

The model should operate inside a policy-defined action space.

if recommendation["route"] not in ALLOWED_ROUTES:
    return deterministic_route(message)

return recommendation["route"]

For more complex routing, policy should filter invalid candidates before optimization or scoring whenever possible:

Candidate destinations
        ↓
Security / residency / workload policy
        ↓
Allowed candidates only
        ↓
Contextual scoring or recommendation
        ↓
Validated selection

This is stronger than asking a model to consider forbidden options and hoping it remembers not to select them.

Guardrail 3: Confidence gating

The reference pattern uses a configurable threshold such as:

MIN_CONFIDENCE=0.7

if recommendation["confidence"] < MIN_CONFIDENCE:
    return deterministic_fallback(message)

But there is an important nuance. An LLM-generated value of 0.91 should not automatically be interpreted as a calibrated 91% probability of correctness. Unless confidence has been independently calibrated and evaluated, treat it as a decision signal, not statistical truth.

The threshold is application policy:

Recommendation
      ↓
Confidence acceptable?
   /             \
 No               Yes
 ↓                 ↓
Fallback        Continue

Confidence gating becomes more useful when combined with validation, policy constraints, and observed performance.

Guardrail 4: Deterministic fallback is a first-class path

Fallback should not be emergency code that nobody tests. It should be a normal operating mode.

In the reference implementation, the architecture supports:

USE_BEDROCK=false

so the message-processing path can operate without model access.

if not USE_BEDROCK:
    return deterministic_decision(message)

try:
    recommendation = invoke_model(context)
except Exception:
    return deterministic_decision(message)

The same fallback can be used when Bedrock is unavailable, invocation times out, the response is malformed, policy rejects the recommendation, or confidence is below threshold.

That creates a useful invariant: loss of AI capability should not automatically mean loss of messaging capability.

Guardrail 5: Bound retry recommendations

Adaptive retry is useful only if the model cannot invent unbounded retry behavior. Suppose the model recommends:

{
  "retry": true,
  "retry_delay_seconds": 7200,
  "confidence": 0.92
}

The application may allow only:

MIN_RETRY_DELAY = 30 seconds
MAX_RETRY_DELAY = 900 seconds
MAX_ATTEMPTS = 5

Policy wins:

delay = max(
    MIN_RETRY_DELAY,
    min(recommendation["retry_delay_seconds"], MAX_RETRY_DELAY)
)

if attempt >= MAX_ATTEMPTS:
    send_to_dlq(message)

Failure class should matter as well. A useful deterministic baseline is:

Transient failure      → Bounded retry + backoff
Permanent failure      → DLQ / terminal handling
Ambiguous failure      → Limited retry → Re-evaluate

AI can help classify ambiguous context, but it should not remove hard retry boundaries.

Guardrail 6: Preserve idempotency

AI does not change a fundamental property of queue consumers: messages can be processed more than once. Any side-effecting consumer should therefore remain idempotent.

A useful pattern is to derive an idempotency key from the producer's stable event identifier and record whether the operation has already completed.

Receive message
      ↓
Check idempotency key
   /             \
Seen             New
 ↓                ↓
Skip          Process
                 ↓
             Record result

Adaptive routing and retry make idempotency more important, not less. A message that changes processing paths must still represent the same logical event.

Guardrail 7: Avoid unstable decisions

Context changes. If the decision engine continuously reacts to tiny changes in queue depth, latency, or health signals, routing can oscillate:

Processor A → Processor B → Processor A → Processor B

That is not intelligence. It is instability.

Useful stability controls include minimum dwell time, hysteresis, rate limits, confidence thresholds, and bounded decision frequency. For example, don't change routes merely because Processor B is 2% healthier for one observation — require a meaningful difference for a sustained period.

New recommendation
       ↓
Materially better?
       ↓
Stable long enough?
       ↓
Policy allows change?
       ↓
Switch

Guardrail 8: Make the decision explainable

For every AI-assisted operational decision, record enough information to reconstruct why it happened:

{
  "message_id": "evt-4821",
  "decision": "PROCESSOR_B",
  "decision_source": "bedrock",
  "confidence": 0.91,
  "fallback_used": false,
  "policy_version": "v3",
  "reason": "Processor A degraded; B is approved and healthy"
}

For routing decisions, also consider recording candidate routes, candidates removed by policy, relevant health/context signals, the selected destination, fallback rationale, model/version identifier, and decision latency.

Observability should answer why did this message take this path? without requiring someone to reproduce the model invocation after an incident.

Guardrail 9: Separate model failure from workload failure

These are different events: model invocation failed and downstream processing failed. Do not collapse them into the same metric.

Track at least:

• AI invocation success/failure
• AI decision latency
• Fallback rate
• Low-confidence rate
• Policy rejection rate
• Message-processing success/failure
• Retry count
• DLQ count

A system can have a healthy model path while the workload is failing, or a failing model path while deterministic processing remains healthy. Operational dashboards should make that distinction visible.

A practical decision flow

Putting the pieces together:

Incoming Message
       ↓
Build Context
       ↓
Is AI enabled? ──No──→ Rules
       │Yes
       ↓
Invoke Model
       ↓
Invocation OK? ──No──→ Rules
       │Yes
       ↓
Parse + Validate
       ↓
Valid? ──No──→ Rules
       │Yes
       ↓
Policy Check
       ↓
Allowed? ──No──→ Rules
       │Yes
       ↓
Confidence Gate
       ↓
Above threshold? ──No──→ Rules
       │Yes
       ↓
Approved Action ──→ Execute

The deterministic path is not separate from the architecture. It is part of the architecture.

What I would test before enabling AI decisions

Before allowing recommendations to affect real message paths, test the system against cases such as:

I would also run the model in shadow mode before giving its recommendations operational effect:

Message
   ↓
Existing deterministic decision ──────► Production action
   ↓
AI recommendation
   ↓
Record only

Then compare: How often did AI agree with existing logic? Where did it disagree? Were disagreements useful? What was fallback frequency? What latency and cost did the model add? Which recommendation categories performed poorly?

Only after measuring those results would I expand operational authority.

The goal isn't maximum AI

A successful design is not the architecture that invokes AI most often. It's the architecture that uses AI where contextual reasoning provides measurable value while keeping deterministic behavior where deterministic behavior is already sufficient.

That may mean 80% deterministic / 20% AI-assisted, or 99% deterministic / 1% AI-assisted. The right percentage depends on the workload.

The principle doesn't change regardless of that ratio:

AI recommends. Policy decides. The messaging platform executes.

What's next

Part 1 introduced the architecture. This article defined the guardrails around it. Part 3 turns those ideas into an implementation using Amazon SQS, AWS Lambda, Amazon Bedrock, and AWS SAM: queues, functions, model invocation, structured decisions, deterministic mode, routing, priority scoring, retry handling, deployment, and testing.

The reference implementation is available here: AI-Assisted Message Queue Patterns on GitHub

The examples in this series are architectural patterns and reference implementations intended for experimentation and learning. Production systems should independently evaluate model behavior, security, latency, cost, failure modes, and workload-specific requirements.