# Decision Engine

A deterministic decision layer for AI systems, built for the **[The Decision Engine](https://build.doo.ooo/challenges/decision-engine)** challenge: *can you build a system that knows when it is allowed to act?*

**Live demo:** https://AliJalil.github.io/decision-engine/ · **90-second version:** open the demo, pick the **Refund Approval** domain, run the ⚠️ *velocity_trap* case first (it will surprise you), then flip through the ✅ / ❓ / 🚨 / 🚫 cases in any domain to see all five outcomes.

No model calls anywhere in this codebase. Every confidence, risk, and cost number is a plain, readable function of the input — that's deliberate, since "real system, not a prompt wrapper" is exactly what's being graded.

## What this is

A proposed action goes in (`{domain, input}`); one of five outcomes comes out: **execute, ask, defer, escalate, refuse** — each with a full signal breakdown (confidence, risk, reversibility, cost-if-wrong, missing evidence) and a step-by-step reasoning trace. Every decision is appended to an audit log with its complete input, signals, and trace — nothing is summarized away after the fact.

Three real domains are wired in, each with its own signal-extraction logic and five example cases (one per outcome): **support ticket triage**, **refund approval**, and **production code deploy**. Fields are editable in the UI, so you can perturb any case and watch the outcome move.

## Architecture snapshot: inputs → signals → decision → audit

```mermaid
flowchart LR
    A["<b>Inputs</b><br/>a proposed action +<br/>domain-specific context<br/>(ticket text, refund request,<br/>deploy metadata...)"] --> B
    B["<b>Signals</b><br/>domain.evaluate(input):<br/>confidence · risk ·<br/>reversibility · cost-if-wrong ·<br/>missing evidence · hard-rule flags"] --> C
    C{"<b>Decision</b><br/>engine.decide(signals):<br/>4 gates, in order —<br/>policy → evidence →<br/>risk/reversibility/cost →<br/>confidence"}
    C -->|hard rule fires| R["🚫 REFUSE"]
    C -->|evidence missing| K["❓ ASK"]
    C -->|risk high & hard to undo/costly| E["🚨 ESCALATE"]
    C -->|confidence below bar| D["⏸ DEFER"]
    C -->|all gates pass| X["✅ EXECUTE"]
    R --> AU
    K --> AU
    E --> AU
    D --> AU
    X --> AU["<b>Audit</b><br/>full input + signals + trace<br/>appended, replayable later"]
```

`domains.js` owns the "signals" column — three domains, each with a `fieldMeta` schema for the UI and an `evaluate(input)` function that does the actual reasoning (keyword classification for tickets, threshold arithmetic for refunds and deploys). `engine.js` owns the "decision" column: four ordered gates, each domain-agnostic, each producing one trace line. `audit.js` owns the last column. Nothing about the decision logic lives in the UI layer (`app.js`) — it only renders what the other three modules compute.

## Why the four gates are ordered the way they are

1. **Policy** (hard rules) always wins — a fraud flag or a failing test suite refuses the action regardless of how confident or low-risk everything else looks. This can't be outvoted by good numbers elsewhere.
2. **Evidence completeness** comes before risk scoring, because you can't meaningfully assess risk with a required field missing (e.g. a refund reason that needs proof, but no photo). This produces **ask**, not a guess.
3. **Risk × reversibility × cost** comes before confidence, on purpose: a system can be *extremely* confident about a high-risk, hard-to-undo action and that confidence should not be enough to auto-execute it. That's what routes it to **escalate** instead.
4. **Confidence** is the last gate — evidence is complete, risk profile is acceptable, but is the engine actually sure? Below the bar, it's **defer**: not urgent, just not autonomous.

## The failure test: a case built to slip past a naive checker

Domain: **Refund Approval**, case **`velocity_trap`** (⚠️ in the picker).

On its own, the transaction looks clean: $45, a "defective item" reason with a proof photo attached, from an account that's been active for over two years. A refund-approval script that only looks at *this order* — which is exactly what a lot of real auto-approval logic does — would approve it instantly.

It's the customer's **6th refund request in 30 days**. That's a real velocity/abuse signal, and it's not visible anywhere in the single transaction — only across the customer's recent history. This engine catches it only because `refundCount30d` is deliberately modeled as a first-class signal that feeds `risk` and pulls `reversibility` down, which is enough to cross the escalate gate (`risk ≥ 0.65` and `reversibility ≤ 0.4`) even though every per-transaction field looks fine.

**Where it still breaks, honestly:** the engine has no way to compute `refundCount30d` itself — it trusts whatever the caller passes in. If the upstream system doesn't track refund velocity per customer (a new integration, or velocity spread across multiple accounts/emails/cards the merchant hasn't linked), this signal is silently unavailable and the engine will confidently execute a refund it should have escalated. A decision layer is only as good as the signals it's handed; that's a real limit, not a solved problem, and no amount of clever gating inside `engine.js` fixes a missing input.

## Bonus signals implemented

- **Easy to run locally:** no build step, no `.env`, no dependencies — see below.
- **Realistic synthetic data:** 15 example cases across 3 domains, each grounded in a plausible real scenario (an enterprise outage ticket, a stolen-card refund attempt, an off-hours tier-1 deploy with no on-call sign-off), not toy placeholders.
- **Honest about limits:** see the failure test above, plus the Out of scope section below.

## Running it locally

```bash
git clone https://github.com/AliJalil/decision-engine.git
cd decision-engine
python3 -m http.server 8000   # or any static file server
# open http://localhost:8000
```

No `.env.example` needed — there are no API keys or external services anywhere in this build.

## Two-year thesis: where decision layers go next (≤300 words)

Today almost every "agent" is a prompt plus tool access: given an instruction, it executes. That collapses two different questions — "can this be done" and "should this be done right now, without a human" — into one model call, which is a lot of why agents overreach. Within two years, a decision layer like this one (confidence, risk, reversibility, cost-of-wrong, evidence completeness, each computed from real signals, not a vibe check) becomes a standard, separately auditable component between "the model wants to act" and "the action happens" — closer to a database's isolation level sitting between a transaction being requested and it committing, than to a system prompt.

The five-outcome vocabulary here (execute / ask / defer / escalate / refuse) will likely become close to a default interface, because it maps onto how competent humans already delegate: not a binary do-or-don't, but sometimes a clarifying question, sometimes "this can wait," sometimes "straight to my manager," sometimes a flat no. Frameworks that only expose do-or-not-do will keep looking naive next to this.

The genuinely hard part — why this stays interesting for years, not months — is that risk, reversibility, and cost-of-wrong are domain knowledge, not something a general layer can infer from an action's shape alone. A $50 refund and a $50 database migration aren't comparable by amount; they're comparable by what happens if you're wrong and how expensive it is to notice and undo. That mapping has to be authored by whoever owns the domain, the way an insurance rate table is specified by an underwriter, not discovered by a model. The decision layers that win won't be the smartest models — they'll be the ones with the richest, most honestly-scoped library of domain risk profiles, and the discipline to say "escalate" when they genuinely don't know.

## Notes

**AI tools used:** Built with Claude (Anthropic), which wrote the engine, the three domain models (including tuning the example cases so all fifteen land on their intended outcome — verified with a small Node script, not by hand), the UI, and this document in one session.

**Key decisions:** (1) Zero model calls, anywhere — every signal is deterministic arithmetic or keyword matching over the input, so every number in a trace can be explained by pointing at a line of code. (2) Four ordered gates rather than a weighted score, because a single blended number can't represent "this one factor alone should stop everything" (a hard policy rule) the way an explicit ordered gate can. (3) A fully static, client-only app — no backend, nothing to deploy but static files, so the live demo needs no server and a clean clone runs immediately.

**Out of scope (by design):**
- **Signal extraction is intentionally simple.** Ticket classification is keyword matching, not real NLU; a production system would swap `classifyTicket()` for a proper classifier while keeping the same `{confidence, risk, ...}` contract with the decision engine — the gate logic doesn't need to know or care how signals were produced.
- **No persistence beyond one browser.** The audit log lives in `localStorage`; a real deployment would write it to an append-only store other services and auditors can query.
- **No learning loop.** Thresholds and per-domain weights are hand-set constants, not calibrated against outcome data. That's a reasonable next step once real decisions and their eventual correctness are being tracked.
- **Domain risk models are illustrative, not authoritative.** The refund/deploy/ticket weightings encode plausible judgment calls (e.g. "6+ refunds in 30 days is suspicious"), not a real company's actual risk policy — swapping in a real one is exactly the `evaluate()` contract this is built around.

## License

MIT