Adithya N Murthy

No. 04of 22LandmarkResearch

Role-specific AI agents whose work is checked before it is certified

AI E (AI Employees)

Five AI employees for finance, data analysis, HR, QA and DevOps on one shared runtime. The language model only drafts: numbers and rules come from deterministic engines, and an answer is marked certified only when those checks pass and a reviewer model from a different family agrees.

When
Oct – Nov 2026
Status
Built

Progress

Progress70%

All five employees, the website, the security layer and the evaluation suites exist and have run. It runs on a local machine only, payments are a demo wallet, and no model is fine-tuned.

What comes next

  1. Re-grade the HR employee's results and save them, so they can be quoted.
  2. Add a human or rival-tool baseline, and a blind set for the data analyst.
  3. Host it publicly once rate limiting, a content security policy and two-factor sign-in are in.

246 / 250

analyst questions answered correctly by Vega, the data analyst (98.4%)

On its own 250-question evaluation set, after several rounds of iteration on that set. Vega has no blind set yet.

Evaluation run output (data analyst, results/full9.log and full9.jsonl)Verified. Backed by a file or public page that has been checked.Evaluation run output (data analyst, results/full9.log and full9.jsonl) — Read in the project audit of 13 November 2026.

0 of 68

certified answers that were wrong, in that same run

Only 68 of the 250 answers earned the certified label. The claim is about those 68.

Evaluation run output (data analyst, results/full9.log and full9.jsonl)Verified. Backed by a file or public page that has been checked.Evaluation run output (data analyst, results/full9.log and full9.jsonl) — Read in the project audit of 13 November 2026.

0 of 554

planted-bad finance documents wrongly marked ready by Aurex

Finance accuracy scorecard (finance ops, results/accuracy_scorecard.md)Verified. Backed by a file or public page that has been checked.Finance accuracy scorecard (finance ops, results/accuracy_scorecard.md) — Read in the project audit of 13 November 2026.

56/56 → 16/20

planted bugs found by Verix: on its development set, then on blind projects

The blind run also made 1 false claim. Tuned scores do not fully carry over to unseen work.

QA scorecards (SCORECARD_development.json and SCORECARD_blind.json)Verified. Backed by a file or public page that has been checked.QA scorecards (SCORECARD_development.json and SCORECARD_blind.json) — Read in the project audit of 13 November 2026.

12 / 12

direct prompt-injection tests blocked by the security layer

One small test set, run on 8 November 2026. In the same run, 79 of 80 pieces of other roles' work were refused.

Security test report (security/reports/tests-20261108.md)Verified. Backed by a file or public page that has been checked.Security test report (security/reports/tests-20261108.md) — Read in the project audit of 13 November 2026.

8.0s

month-end close on 507,745 ledger lines

A load test of the deterministic finance engines on one workstation. No model calls are involved.

Finance load-test output (load_test_2026-10-27.json)Verified. Backed by a file or public page that has been checked.Finance load-test output (load_test_2026-10-27.json) — Read in the project audit of 13 November 2026.

Exhibit 01 · Walk-through

Watch one job

  1. Step 01

    Assign the work

    The request is in plain language: how many customers in the Northwind sample database have never placed an order? List them, then send the report to Slack. Vega, the data analyst, starts: a plan, SQL drafted by two models, a read-only run, checks, a critic, a report.

    Screenshot of the AI E workspace. An assignment about customers in the Northwind database shows as running at 33 seconds, with a note that each step appears as it happens. A side panel lists how this employee works: plan, SQL from 2 models, read-only run, checks, critic, report.
    33 seconds in. The job is running.

  2. Step 02

    It stops and asks

    Sending to Slack changes something outside the system, so the job pauses on an approval card that shows exactly what would be sent. A person chooses Deny or Approve. Nothing risky happens without that choice.

    Screenshot of the same job at 1 minute 11 seconds. A highlighted card reads: Approval needed, send_report, with Deny and Approve buttons and the exact message that would go to Slack: 2 customers in Northwind have never placed an order.
    The approval gate, with the payload in full.

  3. Step 03

    The answer, with the decision on record

    Here the reviewer pressed Deny. The job still finishes, in 1 minute 22 seconds, and the answer arrives with the denial logged above it: two customers have never placed an order.

    Screenshot of the finished job, marked done at 1 minute 22 seconds. A red bar records that send_report was denied; the reviewer's test address is masked. Below it, the answer: 2 customers in the Northwind database have never placed an order, with key points beginning Total customers: 91.
    Done. The test account's address in the red bar is masked.

Exhibit 02 · Compare

Tuned versus blind

Development set

56 / 56

Verix, the QA employee, found every planted bug in the projects it was developed against, with no false positives, and triaged 24 of 24 CI failures. These are the cases it was tuned on.

Blind, never seen

16 / 20

On projects it had never seen, Verix found 16 of 20 bugs and made 1 false claim; its PR review caught 19 of 20. Argus, the DevOps employee, found 20 of 25 issues in a blind cluster sweep, with no false findings on 8 clean objects.

Bar chart titled: tuned scores do not fully carry over to unseen work. Pairs of bars for tuned and blind runs. Verix bug hunt: 56 of 56 tuned, 16 of 20 blind. Verix PR review: 55 of 56 tuned, 19 of 20 blind. Argus sweep: 47 of 47 tuned, 20 of 25 blind.
The project's own chart, from its final reports.

The two sets of numbers are always shown together: tuned scores do not fully carry over to unseen work. Vega, the data analyst, has no blind set yet.

Exhibit 03 · Scorecard

Every verified result

Run outputs read in the project audit of 13 November 2026. Tuned and blind results are listed together. There is no human or rival-tool baseline yet.

Every verified result
MeasureResultNoteSource
Vega (data analyst): 250 analyst questions across 10 domains246 / 250 (98.4%)Its own evaluation set, after several rounds of iteration on it. No blind set exists for Vega.evals/results/full9.log, full9.jsonl
Vega: trap questions70 / 70evals/results/full9.log, full9.jsonl
Vega: wrong answers marked CERTIFIED068 of the 250 answers were certified, and all 68 were correct.evals/results/full9.log, full9.jsonl
Vega: statistical tasks30 / 30evals/results/s2_full7.log
Aurex (finance): 63 golden finance questions63 / 63With Qwen3.6-27B. 62 / 63 after a later build; the miss passed on a re-run.evals/results/2026-10-27_ovhcloud_Qwen3.6-27B_final.json and final2
Aurex: planted-bad documents wrongly marked ready0 of 554evals/results/accuracy_scorecard.md
Aurex: SEC EDGAR 10-K rebuilds and bank-statement files11 / 11 companies, 13 / 13 filesevals/results/accuracy_scorecard.md
Aurex load test: invoice validation5,000 invoices at 349.5 per second400 / 400 recall, 0 / 4,600 false alarms.evals/results/load_test_2026-10-27.json
Aurex load test: month-end close507,745 ledger lines in 8.0 sA 50,000-line bank reconciliation took 8.6 s. Deterministic engines on one workstation.evals/results/load_test_2026-10-27.json
Verix (QA): development set56 / 56 planted bugs0 false positives; CI triage 24 / 24. This is the set it was tuned on.evals/results/SCORECARD_development.json
Verix: blind projects16 / 20 bug hunt, 19 / 20 PR review1 false claim in the bug hunt.evals/results/SCORECARD_blind.json
Argus (DevOps): blind sweep20 / 25 issues found0 false findings on 8 clean objects, in 1.6 s.evals/blind_1/grade_blind.json
Nexa (HR): accent fairness in voice interviews0.0 score spread across 5 accentsAbout 5.9% speech-to-text error. Nexa's question-answering results are not quoted: no graded result file was saved.interviews/evals/results/accents.json
Security: own-role work accepted30 / 30security/reports/tests-20261108.md
Security: other roles' work refused79 / 80security/reports/tests-20261108.md
Security: direct injection blocked12 / 12One small test set, 8 November 2026.security/reports/tests-20261108.md
Security: data-loss checks5 / 5security/reports/tests-20261108.md
End-to-end customer journey18 / 18 answer checks, 14 / 14 steps8.2 minutes in all.outputs/20261023-0122_customer_journey_FINAL/journey.json

01

What it is

A company "hires" an AI employee for a role, connects its own tools, assigns work in plain language and approves anything risky. Five employees exist: Aurex for finance operations, Vega for data analysis, Nexa for HR, Verix for QA and testing, and Argus for DevOps. It was built between 22 October and 9 November 2026 as a startup project, and it runs on a local machine.

02

The hard part

A language model will state a wrong number with complete confidence. So here the model only drafts. Numbers, tax, banking and labor-law facts come from deterministic engines and dated rule packs. An answer is labeled CERTIFIED only when the deterministic checks pass and a reviewer model from a different model family agrees; otherwise it is PROVISIONAL, or the employee asks. The project's own runs show why the family rule matters: a reviewer from the same family certified three wrong answers, and under the independent-family rule all 68 certified answers were correct.

03

What was built

A shared agent runtime: a model gateway with provider fallback, the agent loop, an approval gate, a check layer, an MCP client, a Docker sandbox with no network, an AES-256-GCM credential vault and a hash-chained audit log. A security layer wraps it: a role-boundary judge, prompt-injection defense and data-loss blocking. The five employees sit on top, each with its own tools, from 30-plus finance engines and 56 country tax packs to a 70-rule cluster sweep with dry-run and undo. The website is Next.js 16 with 20 routes and streams jobs live through a FastAPI bridge. In all, about 80,000 lines of Python and 6,000 of TypeScript. The models are hosted and swappable; none is fine-tuned.

04

What was measured

On its own 250-question set, after several rounds of iteration, Vega answered 246 correctly, passed 70 of 70 trap questions, and certified 68 answers, all of them correct. Aurex marked none of 554 planted-bad finance documents as ready, and closed a month of 507,745 ledger lines in 8.0 seconds. Scores drop on unseen work, and the project reports both: Verix found 56 of 56 planted bugs on its development set and 16 of 20 on blind projects, with one false claim. Argus found 20 of 25 issues in a blind sweep. The security layer blocked 12 of 12 direct injection tests and refused 79 of 80 pieces of out-of-role work.

05

What it does not claim

It is not hosted publicly, payments are a demo wallet, and no model has been fine-tuned. Vega has no blind test set yet, so its 98.4% is a tuned figure. The HR employee's question-answering results are not quoted, because no graded result file was saved. There is no human or rival-tool baseline. The project's paper, report and deck are drafts and are not linked.

Figures

Dark landing page of the AI E website. The headline reads: An AI workforce for every role and every task. Below it, four illustrated cartoon portraits of AI employees, and buttons to hire your first employee or see them work.
Landing page of the local website. It listed four of the five employees when this was captured.
Flow diagram titled: the model drafts, code and an independent reviewer decide the label. A customer's question goes to two drafts from two model families, then deterministic checks that run the SQL and math in code, then an independent critic from a third model family. Three outcomes: certified when all checks and the critic agree, provisional when something is unconfirmed, or ask and abstain when unclear or not allowed.
The central idea, from the project's own figures. Its footnote: a same-family reviewer certified 3 wrong answers; with the independent-family rule, 68 of 68 certified answers were correct.
Layered architecture diagram, built versus planned. Built: a web app, terminal and candidate interview page; an API bridge; a security layer; the five AI employees Aurex, Vega, Nexa, Verix and Argus; a shared runtime with the agent loop, the check layer and approval gate, deterministic engines, and the model gateway with audit log. Drawn dashed as planned and not built: a desktop app, a payments provider, public hosting, more roles and fine-tuned models.
System architecture, from the project's own figures. Dashed boxes are planned, not built.
Two charts of Vega's seven evaluation runs on 250 questions. Accuracy per run: 95.2, 98.0, 94.8, 98.4, 96.0, 95.6 and 98.4 percent. Below, wrong answers that were still certified: 1 in the third run, 3 in the fifth, and 0 in the final clean run. A footnote says one run happened before Vega's memory was found to be contaminated by earlier test runs.
Vega's accuracy, run by run, from the project's own report. It went down as well as up; the last run is the first with an empty memory and an independent-family reviewer.

One question

The answer is on this page. Get it right and it goes in your passport on the map.

Challenge

Vega, the data analyst, answered 246 of 250 questions correctly. How many of its answers earned the CERTIFIED label?

Show the answer

68. 68. An answer is certified only when the deterministic checks pass and a reviewer from a different model family agrees, and all 68 certified answers were correct. Source: the run's output files (full9), read in the project audit of 13 November 2026.

Stack
  • Python
  • FastAPI
  • Next.js 16
  • React 19
  • Tailwind 4
  • PostgreSQL
  • DuckDB
  • Docker
  • Kubernetes (kind)
  • Prometheus
  • Model Context Protocol (MCP)
  • Whisper
  • ONNX Runtime
  • Hosted and local LLMs (OpenRouter, Groq, Gemini, Ollama)