LeakFinder

Built at the CreateOS × Stripe Projects hackathon

An AI agent that finds money leaking between Stripe and your database.

It writes its own code and runs it in a sealed CreateOS sandbox.

For a SaaS founder whose Stripe account and database disagree about who is paying.

$npx leakfinder audit
npm package in preparation
View the code on GitHub How it works

Until the package is on npm, run it from the repo. The README has the steps. What has run live so far is listed under Status.

Demo dataset · answer key

Planted by the seed

$348/ month leaking

Vercel's nextjs/saas-starter, its schema unmodified. 252 users, Pro at $29 a month. Nobody tells the agent this schema; it has to find all of it.

  • APro access nobody pays for7 never paid, 5 cancelled in Stripe12
  • BPaying with no accountStripe charges them, the app has no user3
  • CRows out of syncThe app says active, Stripe says canceled5
12 × $29 = $348 a month. These are the planted numbers, not a run result.

How it works

Three things to check

Give LeakFinder a Postgres and a Stripe key. The agent works out your schema, writes the reconciliation, and writes the fix.

1

It writes and fixes its own code, in a sealed room.

The agent has never seen your schema. It writes Python, runs it in a CreateOS micro-VM and reads what comes back. When a query fails, it reads the error and rewrites the script. Nobody is typing.

The VM can reach api.stripe.com and your database host. Everything else is dropped by a rule on the host, outside the VM, so code inside cannot change it.

  • api.stripe.comallowed
  • your database hostallowed
  • anything elsedropped
2

The model never sees a customer.

It gets code, schema, counts and errors. Emails are redacted before any text reaches it, and the findings go to a file inside the VM, not to the model. Every byte the model was shown is saved in a transcript you can grep.

Take a customer email from the report and search for it in that transcript. The count is the proof. Email addresses are removed before the model sees anything; other fields stay out because the agent's scripts are asked to print counts only.

# an email from the report
$ grep -c "<email>" transcript.jsonl
0

The demo will run this check with an email the seed plants.

3

Apply closes the leak, in Stripe test mode.

The agent writes the fix script too. It dry-runs first and prints a plan with counts. Nothing changes until a person approves it: invoices out, refunds issued, access revoked, rows synced.

Then a separate program reads Stripe and Postgres directly, without the agent. After the agent's fix ran from the host in test mode, it read zero. Apply runs with a Stripe test key only. Invoices are sent, not collected.

  1. Dry runA plan with countsNothing changes in Stripe or the database.
  2. ApplyInvoices, refunds, revokes, syncsIn Stripe test mode, after a person approves.
  3. VerifyZero, zero, zeropnpm verify:starter counts the drift again from Stripe and Postgres.

The sealed room

The code runs next to the data. The model stays outside.

Your laptop runs the LeakFinder CLI. It passes Python into a CreateOS micro-VM and passes back only what the script prints, with emails redacted, to the model.

The sealed room The model, outside the VM, exchanges text with your laptop: it receives code, schema, counts and errors, and sends back Python. Your laptop runs the LeakFinder CLI, redacts emails and keeps the transcript. The laptop sends the script into a CreateOS micro-VM and gets counts and errors back. The VM has an egress allowlist: api.stripe.com and the database host are allowed, anything else is dropped. Findings go to a file inside the VM and are never sent to the model. The model the model, outside the VM sees no customer rows code, schema, counts, errors Python Your laptop LeakFinder CLI redacts emails keeps the transcript Python script counts, errors EGRESS ALLOWLIST CreateOS micro-VM the agent's Python runs here findings.json never sent to the model rule enforced on the host api.stripe.com allowed database host allowed anything else dropped The sealed room The model, outside the VM, receives code, schema, counts and errors from your laptop and sends back Python. Your laptop runs the LeakFinder CLI, redacts emails and keeps the transcript. It sends the script into a CreateOS micro-VM and gets counts and errors back. The VM can reach api.stripe.com and the database host; anything else is dropped. Findings go to a file inside the VM and are never sent to the model. The model the model, outside the VM sees no customer rows code, schema, counts, errors Python Your laptop LeakFinder CLI redacts emails · keeps the transcript Python script counts, errors EGRESS ALLOWLIST CreateOS micro-VM the agent's Python runs here findings.json never sent to the model rule enforced on the host api.stripe .com allowed database host allowed anything else dropped
The model receives only code, schema, counts and errors. Customer rows never reach it.

What it finds

Three ways Stripe and your database drift apart

A missed webhook is enough. Stripe cancels someone and the app never hears, or someone pays and never gets an account.

DriftWhat is wrongIn the demo datasetWhat Apply does
APro access nobody pays for The app grants Pro. Stripe has no active or trialing subscription for that team. 12 teams7 never paid, 5 cancelled in Stripe while the app never heard Revokes the access. The 7 who never paid get a one-month invoice, due in 7 days; the 5 who cancelled are never billed.
BPaying with no account Stripe has an active subscription for a customer no app user owns. 3 payerscharged every month, with no account to use Refunds the latest payment and cancels the subscription.
CRows out of sync The app's row says the subscription is active. Stripe says canceled. 5 rowsthe same 5 teams Stripe cancelled Sets the app row to Stripe's status.

Pro is $29 a month in the demo dataset: 12 × $29 = $348 a month leaking. These are the seed's answer key. A separate check, pnpm verify:starter, recounts them from Stripe and Postgres.

Built with

Five pieces, one terminal

Stripe ProjectsSets up the stack from the terminal: CreateOS and Postgres, credentials in .env.
CreateOS SandboxThe micro-VM where the agent's code runs, with an egress allowlist.
Stripe API, test modeReads every subscription. Apply sends invoices, refunds and cancellations.
Neon Postgres via Stripe ProjectsThe app's database: users, teams and the plan each team is on.
DeepSeekThe model writes the Python and rewrites it when a query fails. A Claude provider is in the code but has not been run live.

Status

What is real today

  • A prototype, built in one day at the CreateOS × Stripe Projects hackathon.
  • Run live: the agent loop on DeepSeek against the seeded Neon database and Stripe in test mode, with its Python run on the host, not in the sandbox. It found the planted 12, 3 and 5 rows and the $348 a month, run after run.
  • Run live: the agent's fix script, applied from the host in Stripe test mode. pnpm verify:starter then read 0, 0, 0, and a second apply changed nothing.
  • Run live: a CreateOS sandbox with the egress allowlist. A probe to a host outside the list got no answer, and the report it served answered 200 with its token and 403 without.
  • Not run live yet: the full audit inside the CreateOS sandbox, and Apply from there or from the report's Apply button.
  • The schema is nextjs/saas-starter's, unmodified. The 252 users and the drift are planted by our seed. There are no customers.
  • Built for two schemas in this repo: nextjs/saas-starter's and a users, subscriptions and entitlements one. No others tested.
  • Money moves in Stripe test mode only.