How to QA a SaaS You Built with AI Before Launch

In 2025, security researchers scanned 1,645 apps built with Lovable and found 170 of them, about one in ten, serving their databases to anyone who asked. The flaw got its own identifier, CVE-2025-48757, and the postmortems all share a detail worth sitting with: those founders did not skip QA. They clicked through their apps, everything worked, they shipped. "Works when I click it" and "holds up when a stranger, a crawler or an attacker shows up" are different claims, and only the first one gets tested by default.

So here is how to QA a SaaS you built with AI before launch: what to test with your own hands, what to hand to a testing agent, what to run a crawler over, and in what order. It assumes you built with Lovable, v0, Cursor or Claude Code, and that you would rather spend one afternoon on this than spend launch week answering a bug thread.

Why the demo passing proves almost nothing

Vibe coding is building software by describing what you want to an AI tool and accepting the code it writes. The output carries a specific bias: the happy path gets built and polished, because that is what you asked for and what you check, while everything around it (the wrong password, the expired card, the URL nobody links to) gets whatever the model generated that day.

The bias is measurable. Veracode ran 80 coding tasks across more than 100 language models and the generated code introduced a security vulnerability 45% of the time, while looking functionally correct throughout. Working code and unsafe code, same file. The public side of the ledger looks no better: of the first 37 AI-built sites Tabkeel's exam crawled, 34 launched carrying at least one surface defect, and the middle of the pack carried five.

The question is not whether your app has holes. Statistically it does. The question is which layer they sit in, and whether you find them before your first hundred visitors do.

The Two-Layer QA Model

Every failure an AI-built SaaS ships with lands in one of two places, and the two places need different tools. That split is the whole method, so it gets a name: the Two-Layer QA Model.

Builders test Layer 1, imperfectly, and rarely think about Layer 2 at all, because a browser tab with you logged in looks nothing like what a crawler receives. Half the fronts that decide whether anyone finds or trusts your product are invisible from inside your own session. The model exists to force one question per layer: for Layer 1, "what happens when the input is hostile?", and for Layer 2, "what does a machine that has never met me actually see?"

Layer 1: the app only you can test

Honesty first: Tabkeel does not check this layer. The exam reads your site the way an outsider does, and your business logic is invisible to it. What follows is simply the shortest manual pass that catches the Layer 1 failures that recur in vibe-coded launches, no product attached.

Auth beyond the happy login

Try the wrong password. Request a password reset and confirm the email arrives and the link works. Then the test almost nobody runs: log out, copy the URL of a page that requires an account, and open it in a private window. If it renders, it renders for the whole internet.

Data rules, not data

Create two test accounts. Logged in as the first, try to read the second one's records, by guessing an ID in the URL or replaying a request with a different identifier. Missing Row Level Security was the root cause behind CVE-2025-48757, and it fails silently: the app looks identical whether the rules exist or not, right up until someone checks.

Payments, including the ugly parts

Run a purchase with Stripe's declined-card test number, not just the success one. Then cancel the subscription from the Stripe dashboard and confirm your app noticed: webhooks are where AI-generated billing code quietly does nothing, and a customer who cancels but keeps full access is revenue you will never see again multiplied by everyone who finds the trick.

Failure out loud

Every action needs a visible failure state, and every screen needs a sensible empty one. Open a brand-new account: what does the dashboard show with zero data? Search for something that does not exist. Submit a form with the network throttled to failure. A vibe-coded app that fails silently trains users to distrust it faster than one that breaks loudly.

That pass takes about ninety minutes by hand. AI testing agents such as Autonoma or VibeEval can then generate and run functional tests from your codebase, which is worth it for flows you will not re-test manually every release. What no Layer 1 tool does is step outside the app and read your site the way the public does. That part is Layer 2.

Layer 2: the surface everyone else reads

The surface is everything reachable without an account, plus every machine reading it. Tabkeel's exam covers this layer as seven fronts, from AI readability to billing integrity, and the field-by-field version is the 7-front checklist for AI-built sites. What matters here is what a surface failure looks like, and why nothing in Layer 1 tooling ever sees one.

Take the most common specimen. A soft 404 is a page that shows a human "not found" while returning HTTP 200, so machines file the error page as real content. AI frameworks produce them constantly: the client router catches the bad route and renders a pretty fallback, and the server never hears about it. Google then indexes your error page, and your real pages compete against it. When the exam catches one, the finding arrives with the failing URL as clickable evidence, and the correction comes written as a prompt for the same agent that built your site. This is the exact fix text the exam returns for that finding:

A nonexistent route returns HTTP 200 (soft 404) instead of 404. Make the server return status 404 for routes that do not exist, otherwise Google indexes your error pages.

Paste that into Cursor or Claude Code and the bug class dies in one commit. The other fronts follow the same shape, evidence first, then a paste-ready prompt: a robots.txt that blocks GPTBot while your roadmap says "be visible to AI", a pricing page charging $29 where the terms still say $39, a privacy policy promising no tracking on a site that loads three trackers.

Cost of checking, measured honestly: while writing this guide we pointed the free missing-states test at PromptEval, a production SaaS, and it read 25 pages in 2.8 seconds and came back clean on that front. Clean is a real result too. The point is that Layer 2 verification costs seconds, which removes the last excuse for skipping it. And the surface extends beyond your own pages: the exam also asks ChatGPT what your product is and compares the answer against what your site claims, because the answer is wrong more often than founders expect. We measured exactly how often across 48 startups we asked ChatGPT about: 44 got a wrong or empty answer.

Which tool catches which failure

None of the tool classes in this space compete with each other, whatever their landing pages imply. They cover different layers, and a launch that skips a layer is betting nobody looks there. Here is the map, blind spots included.

Pick one from each layer and you are ahead of the 92% of AI-built sites in our corpus that shipped with at least one unexamined surface finding. For the second layer, running the exam on the free tier crawls 25 pages of a site, shows every finding on all seven fronts with its evidence, and writes out the fix for the most severe one; the Founder plan writes all of them and re-asks ChatGPT about your product weekly so you see the answer move as you fix things.

How to test a vibe-coded app: the launch-week order of operations

Production readiness is the state where a stranger, a crawler and a paying customer can each do their job on your site without you in the room. Getting there from a working demo takes one afternoon, in this order:

1. Freeze features. QA on a moving target measures nothing. Whatever is not built by QA day ships in week two.
2. Run the Layer 1 manual pass. Auth, data rules, payments, failure states, as above. Ninety minutes, two test accounts, one private window.
3. Point a testing agent at the repo. Let it generate the regression suite while you do other work; keep whatever tests map to money or data.
4. Run the surface exam. Crawl the staging or production URL. Read the findings against their evidence, paste each fix prompt into your agent, redeploy.
5. Re-run and diff. The second exam should show findings resolved, not just fewer. Anything persisting is either accepted debt (write it down) or a fix that did not take.
6. Connect Search Console and wait. Data starts accruing from day one, and two weeks in you can start improving your CTR once Search Console has data instead of guessing.

The order matters because each step feeds the next: the manual pass finds what the agent should regression-test, the exam findings become the commits, the diff proves the commits worked. QA a SaaS you built with AI in that sequence once, and the re-run before every major release drops to minutes, because everything is either automated or evidenced.

Frequently asked questions