Self-Hosted Visual Regression Testing: BackstopJS vs. Argos (and What Happened to Lost Pixel)

Picture standardizing your CI on one of the three open-source visual regression tools everyone shortlists (BackstopJS, Argos, Lost Pixel) and discovering, a week into the spike, that two of the three aren't what they looked like from the outside. Argos has excellent code on GitHub, but its maintainer says plainly that self-hosting is neither supported nor documented. Lost Pixel was archived on April 22, 2026, when the team joined Figma, and shipped no migration path for the teams already running it.

That leaves exactly one of the three that is genuinely self-hostable and maintained today. Which is a strange thing to say about a category with this much apparent choice, but it's where the landscape actually sits. The tools people treat as three equals are, in practice, one working self-hosted option, one strong tool that lives on someone else's servers, and one you should be migrating off.

This is a self-hosting-first read on the self-hosted visual regression testing tools that survive contact with reality: which one fits which constraint, and what to do if you're on Lost Pixel right now.

The Short Version

• BackstopJS is the self-hostable, maintained, MIT-licensed, zero-cost option. Puppeteer by default with Playwright support, pixel diffing via Resemble.js, and an HTML review UI (with Docker rendering caveats you need to plan around).
• Argos is the stronger tool on workflow (PR review, and ARIA-snapshot diffing since late 2025), but it is a managed SaaS in practice. The maintainer provides no support or documentation for running it yourself.
• Lost Pixel was archived in April 2026. Do not adopt it for new work, and if you're on it, plan your migration now.

The one-line verdict: for a fully self-hosted CI pipeline, BackstopJS is the practical choice. If you want the best review workflow and can accept a SaaS dependency, Argos is the better tool.

(What this doesn't cover: Cypress-based visual testing, Storybook-specific component workflows, and hands-on benchmark numbers. This comparison is built from primary-source facts, not a test harness I ran myself.)

The Self-Hosted Visual Regression Landscape Changed

Two specific events in the last year reshaped this shortlist, and both of them are easy to miss if you're working from a listicle written before they happened.

The first: in a GitHub discussion in September 2025, an Argos maintainer stated that Argos provides no support or documentation for running it outside of the managed service. The code is MIT-licensed and sitting in the open, so "you can self-host it" is technically true, but there is no supported path, a distinction that matters enormously once you're the person on call for it.

The second: the Lost Pixel repository was archived on April 22, 2026, read-only, when the team announced they were joining Figma. No migration guidance accompanied the sunset. A tool that was, until recently, a legitimate free self-hosted alternative to Percy and Chromatic is now a dependency with no maintainer.

Both changes point the same direction. The reason teams end up here in the first place, SaaS pricing that grows with browsers times viewports times pages, or a compliance rule against uploading screenshots of unreleased UI to a third party, hasn't gone away. What's changed is that the self-hosted menu is shorter than it looks.

Section key takeaway: Only one of the three tools people shortlist is genuinely self-hostable and actively maintained today.

BackstopJS: The Genuinely Self-Hostable Option

Install is npm install -g backstopjs (or a local project install), and that single fact is most of why BackstopJS wins the self-hosting axis: there is no service to reverse-engineer, no managed backend it secretly depends on. It runs as an npm package or a Docker container on infrastructure you own, under an MIT license, at zero licensing cost. The README notes the current line (6.3.x) supports Node 20.

Under the hood it drives Puppeteer by default (headless Chrome) and also supports Playwright across chromium, firefox, and webkit. Diffing is pixel-based via Resemble.js, with a configurable misMatchThreshold value that controls how much pixel drift counts as a failure. Approvals happen through an HTML report with an in-browser UI for accepting changes.

The limitations are where a strategic decision gets made. BackstopJS has no built-in PR-blocking workflow: it will tell you a diff exists, but wiring that into a "this PR cannot merge" gate is on you. Its configuration lives in JSON, one scenario at a time, and at a few hundred test cases that JSON becomes a maintenance surface of its own. Development is slow-moving rather than fast-moving: the project is not archived, the package currently sits at version 6.3.25, and the repo shows roughly 7,200 GitHub stars and 516 open issues. So treat BackstopJS as mature and usable, not as a fast-evolving visual-testing platform.

The sharpest operational risk is Docker rendering. Two issues, white or cut-off screenshots in Docker (#1156) and inconsistent results after the v5.3.0 Docker upgrade (#1303), have been open for years without resolution. Docker is also the thing you want for CI, because it normalizes rendering across machines. So the tool that gives you the most control also hands you a known, unfixed flakiness class in exactly the setup you'll run it in. That's a plannable risk, not a dealbreaker, but it needs to be on the table before you commit.

Pro tip, the cross-OS baseline trap: if your developers capture baselines on macOS and your CI runners are Linux, your tests will fail on font anti-aliasing differences alone, before any actual regression exists. As one practitioner has documented, font rendering differs between the Linux image used on CI services and a local macOS machine, enough to fail a visual test on its own. The fix is to generate and compare baselines in one environment (run BackstopJS inside the same Docker image locally that you use in CI), so the reference and comparison images come from identical rendering.

Argos: The Best Workflow, But SaaS in Practice

Argos gets the review experience right in a way BackstopJS doesn't, and that's the whole reason it's on this list despite the self-hosting caveat. It integrates with Playwright, Cypress, Storybook, WebdriverIO, and Puppeteer, and plugs into GitHub Actions, GitLab CI, Bitbucket, and Azure DevOps. The PR-review workflow is a first-class product surface, and it has kept moving. Recent changelog entries add collaborative reviews with pinned comments (June 2026) and auto-ignoring of flaky changes.

Its most interesting technical differentiator is diffing. On top of deterministic pixel diffing, Argos added ARIA-snapshot diffing for its Playwright SDK on November 4, 2025. ARIA-snapshot diffing compares the accessibility tree (roles, labels, and structure) alongside the pixel screenshot, which catches a class of semantic and accessibility regressions that a pure pixel diff can miss (a button that still looks identical but lost its accessible label, say). For teams with accessibility commitments, that's a genuine capability the pixel-only tools don't offer.

Then the constraint. The code is MIT, but the product is the managed service, and the maintainer has been explicit that self-hosting is undocumented and unsupported. You can stand it up from source. You just do so knowing that any upstream change can break your deployment and no one is obligated to help. There is a related uncertainty worth flagging rather than asserting: community reports suggest GitLab Self-Managed integration expects an Enterprise plan (I'm treating that as a community signal, not a confirmed spec, so verify it against your own setup before it drives a decision).

This produces an uncomfortable gap. A team with strict privacy requirements (the exact team most motivated to self-host) that also wants the Argos review experience has no good path. They can use the managed service and accept the third-party dependency, or self-host without support, or pick a different tool. There isn't a clean fourth door, and it's more honest to name that than to pretend the open-source license closes it.

Section key takeaway: Argos is the stronger tool. Self-hosting it is unsupported and operationally fragile.

Lost Pixel: Archived, and What to Do If You're On It

The Lost Pixel repository's own banner is the whole story in one line: the repository was archived by the owner on April 22, 2026, and it is now read-only. The team joined Figma, the last release was v3.22.0 in November 2024, and no migration guidance was published.

Until the archival, Lost Pixel was a reasonable pick (Docker-based rendering, Playwright and Storybook support plus Ladle and Histoire, and both full-page and component-level visual testing). None of that changed technically on April 22. What changed is that it now has no maintainer, no security patches, and no roadmap, which for a CI dependency is disqualifying for new work regardless of how well the current version runs.

If you're on it today, the migration splits by how you were using it. If you relied on its component-level testing, especially Storybook or Ladle workflows, your closest replacement is the Argos managed service or Playwright's native screenshot testing. BackstopJS can capture selectors or Storybook URLs, but it does not give you the same built-in component workflow Lost Pixel did, so you would need to model those component states yourself. If you used it for full-page visual regression, BackstopJS is the direct self-hosted successor. Either way, the move is not urgent in the "your builds will break tomorrow" sense, but it is on the clock: an unmaintained testing tool quietly rots as your browser, OS, and framework versions drift past what it was last built against.

Comparison at a Glance

Here's the same information as a side-by-side, so you can scan the axes that matter for a CI decision before mapping them onto your own constraints.

How to Choose (Decision Framework)

Start from your hardest constraint, not from a feature list. The constraint eliminates most of the menu for you.

• Fully self-hosted, zero budget, and you can accept full-page-only testing with manual review: BackstopJS. It's the only option that satisfies self-hosted and maintained and free at the same time, and the trade-offs (no PR gate, JSON config, Docker flakiness) are known quantities you can design around.
• You want the best review experience, PR gating, and ARIA diffing, and you can accept a SaaS dependency: the Argos managed service. This is the path to the strengths of Argos without the unsupported-self-hosting risk. If a compliance rule forbids third-party screenshot upload, this door is closed and you're back to BackstopJS.
• Small test suite or Playwright-first: consider adding no dedicated tool at all. Playwright ships toHaveScreenshot() natively, and for suites below roughly 100 tests it may be all you need. One practitioner guide puts the inflection point where a dedicated tool starts earning its keep at around that scale (treat that as an industry observation, not an official Playwright figure). The management overhead of baseline files in git grows past that point, which is where a dedicated tool pays for itself.
• Currently on Lost Pixel: migrate now, per the split in the previous section. Do not treat "it still works" as a reason to stay.

One caveat that predates all of these tools and outlives them: the Sparkbox team, writing years ago, chose a hosted option specifically because getting engineers to actually use the tests was harder than the setup. A self-hosted tool with a weaker review UI can win on cost and lose on adoption (if your team ignores a noisy BackstopJS report, its zero licensing cost bought you nothing). Factor the review experience into the decision, not just the price.

Section key takeaway: The constraint picks the tool. Self-hosted-and-free points to BackstopJS, best-workflow-and-SaaS-acceptable points to Argos, and small-and-Playwright-first may point to no tool at all.

Sizing the CI Runner for Headless Chromium

All three tools drive headless Chromium, and the memory footprint of Chromium (not CPU) is what sets your runner size, because each concurrent browser worker wants its own few gigabytes of RAM. Undersize the box and your visual tests don't fail cleanly. They get OOM-killed mid-run and report as flaky infrastructure noise, which is the worst kind of failure to debug.

As an approximate guide, sized by parallelism rather than as hard figures (actual usage depends on page complexity and how many workers you run in parallel):

• Small team or serial tests: around 2 vCPU / 4 GB RAM handles a single headless Chromium worker with CI daemon overhead.
• Team CI use: around 4 vCPU / 8 GB RAM supports roughly 2 to 4 parallel browser workers, which is where most teams land.
• High throughput or large suites: around 8 vCPU / 16 GB RAM for high parallelism, or when the visual regression run shares a runner with other CI work.

Storage is the quiet line item. Screenshot artifacts and historical baselines accumulate run over run, so plan for that growth: budget generous storage headroom if you keep baselines locally, or push baselines and artifacts to object storage so the runner disk isn't the constraint.

A VPS is the natural host for a self-hosted CI runner doing this, because you want predictable, dedicated RAM for the Chromium workers, and a homelab box only makes sense if the hardware is already sitting there. If you're standing up the surrounding CI stack too, the pieces around the visual-testing tool are one-click deployments in the Cloudzy marketplace: Gitea, GitLab, Jenkins, and Forgejo for the CI forge, and Docker for the rendering consistency the tools need. The visual regression tool itself installs on top via npm or Docker, so spin up a Linux VPS sized from the ranges above and run it there. If you're wiring up Docker-based CI, the Cloudzy guides on self-hosted GitLab alternatives and listing Docker containers cover the adjacent setup.

Frequently Asked Questions

Is Argos Self-Hostable?

Technically yes (the code is MIT-licensed), but the maintainer stated in September 2025 that Argos provides no support or documentation for running it outside the managed service. Treat self-hosting as unsupported and operationally fragile: it can break on any upstream change with no maintainer obligation to help. For most teams, the managed service is the practical way to get the benefits of Argos.

What Happened to Lost Pixel?

Lost Pixel was archived on April 22, 2026, when the team joined Figma and sunset the product. The repository is read-only, the last release was v3.22.0 in November 2024, and no migration guidance was provided. Do not adopt it for new work. If you're using it, plan a migration to BackstopJS (full-page testing) or Argos and Playwright-native (component testing).

What Is ARIA Snapshot Diffing?

ARIA-snapshot diffing compares the accessibility tree (roles, labels, and structure) alongside the pixel screenshot, catching semantic and accessibility regressions that a pixel diff alone can miss, such as a visually identical button that lost its accessible label. Argos added the feature on November 4, 2025.

How Much RAM Does Headless Chromium Need in CI?

A single serial worker typically runs in about 2 to 4 GB, and concurrent workers scale RAM roughly linearly from there. Actual usage depends on page complexity and how many browser instances you run in parallel, so treat these as guides rather than hard figures and leave headroom to avoid OOM-killed test runs.

Does BackstopJS Support Playwright?

Yes. Puppeteer is the default engine (headless Chrome), and BackstopJS also supports Playwright across chromium, firefox, and webkit. You select the engine in your configuration.