Auditing PWAs After Lighthouse Dropped the PWA Category¶

Lighthouse 12.0.0, released on April 22, 2024, removed the Progressive Web App category, and Chrome 126 DevTools shipped without it. There is no replacement score. Chromium's own installability check is still there, in DevTools and in the DevTools Protocol, and everything else the category covered now needs a small script and a checklist. This post explains why the category went, where each of its audits ended up, and what a PWA audit looks like now. The full, maintained workflow, with custom Lighthouse audits, user flows and CI configuration, is on Lighthouse & Auditing.

What the PWA category used to check¶

The category went through several designs before it was removed. The short version:

By Lighthouse 11.7.1, the last release that had it, the category held six scored audits (installable-manifest, splash-screen, themed-omnibox, content-width, viewport, maskable-icon) and three manual reminders (pwa-cross-browser, pwa-page-transitions, pwa-each-page-has-url). Nothing tested offline behavior, nothing checked that the service worker did anything useful, and nothing looked at Safari or Firefox beyond the manual reminders. By 2024 the category was mostly Chromium's installability check plus four cosmetic audits. Lighthouse & Auditing has the full audit-by-audit history.

Why Lighthouse removed it¶

The Lighthouse 12.0.0 release notes give a one-line reason: "As per Chrome's updated Installability Criteria, Lighthouse has removed the PWA category." They point users to the Chrome DevTools documentation for PWA debugging instead. The Chrome post that announced the new criteria, Revisiting Chrome's installability criteria, says the same: "we have decided to remove this category from Lighthouse", while developers can "still find the checks for optimizations and debugging for installable experiences on DevTools."

The underlying change is this:

• Chrome 108 on Android and Chrome 112 on desktop stopped requiring a service worker with a fetch handler "for installation from the menu". Chrome shows a default offline page for installed apps that don't provide their own.
• The post noted that the promotion algorithm, the one that fires beforeinstallprompt and shows install UI on its own, still required a fetch handler at that time. Current Chromium no longer does, as Installability Criteria documents with tests against Chrome 153.

Three consequences made the category hard to keep:

1. The main audit measured a moving browser rule, not quality. installable-manifest asked Chromium whether the page could be installed. When Chromium lowered the bar, a green badge started to mean "has a manifest with a name, a start URL, a display mode and an icon of at least 144 px" (Chromium's kMinimumPrimaryIconSizeInPx; Google still documents 192 px and 512 px). That is useful to know, but it doesn't say much about how good the app is.
2. It encouraged the wrong fixes. For years the easiest way to get the badge was an empty fetch handler, which gives users nothing and costs a worker startup on every navigation. The Chrome post mentions this directly as a reason for the criteria change. Pitfalls & Anti-Patterns explains the cost.
3. "PWA" isn't a Chromium-only property. Safari has never used a manifest-based installability rule on iOS, and since Safari 26.0 (September 15, 2025) WebKit says "there are now zero requirements for 'installability' in Safari". Firefox on Windows can pin any site to the taskbar since Firefox 143. A single score computed in one engine couldn't represent that, and the manual pwa-cross-browser reminder was the category admitting it.

Where each audit went¶

Most checks still exist somewhere. This table maps every audit from the final PWA category, plus the HTTPS checks that sat next to it, to its fate in Lighthouse 12 and 13 and to what you should use now.

The viewport audit's move in 12.0 came from the SEO reorganization in the same release, not from the PWA removal; Lighthouse 13.0.0 (October 10, 2025, shipped in Chrome 143 DevTools) then replaced many performance audits with performance insights, which is why viewport-insight exists and viewport doesn't.

Two things that break when you upgrade¶

A default Lighthouse run never tests your service worker. Lighthouse's default clearStorageTypes setting includes service_workers and cache_storage, so before each navigation it unregisters your worker and deletes Cache Storage. Every default run is a first visit. That was also true while the PWA category existed, which is one reason its offline audits were unreliable. To measure the repeat visit your PWA is optimized for, use a user flow whose second navigation keeps storage; Lighthouse & Auditing shows how, and why emulating offline inside a Lighthouse run doesn't work.

Old Lighthouse CI assertions fail. Assertions on audits that no longer run don't pass silently: LHCI reports them as failed auditRan assertions (expected 1, actual 0). Configs like this one break on upgrade:

module.exports = {
  ci: {
    assert: {
      assertions: {
        'categories:pwa': ['error', { minScore: 1 }],   // category no longer exists
        'installable-manifest': 'error',                  // audit no longer exists
        'maskable-icon': 'warn',                          // audit no longer exists
      },
    },
  },
};

Delete those assertions rather than setting them to off, so nobody thinks they still check something, and move the checks into a script like the one below. PageSpeed Insights runs Lighthouse too, so its reports lost the PWA section in the same weeks. Lighthouse 13 also requires Node.js 22.19 or later, so an older CI image fails before any audit runs.

What a PWA audit looks like now¶

A PWA audit in 2026 answers five questions, and no single tool answers all of them:

flowchart TD
    A["1. Is it installable in Chromium?"] --> T1["DevTools Manifest pane<br/>Page.getInstallabilityErrors"]
    B["2. Is the manifest complete for every platform?"] --> T2["CI script: members, icons, screenshots"]
    C["3. Does the service worker work: control, offline, updates?"] --> T3["E2E tests with offline emulation"]
    D["4. Is it fast, accessible and well-behaved?"] --> T4["Lighthouse CI<br/>cold and warm runs"]
    E["5. Does it work in Safari and Firefox?"] --> T5["Manual test plan on devices"]
    T1 --> R["Release gate"]
    T2 --> R
    T3 --> R
    T4 --> R
    T5 --> R

Start with a DevTools pass in Chrome or Edge: Application > Manifest shows parser warnings and the Installability section (the same data installable-manifest used to show), the computed app ID, and a maskable-icon safe-area preview that is more useful than the old maskable-icon audit. Browser DevTools walks through every pane.

Then script the Chromium check so it runs on every pull request. The installability audit was always a wrapper around two DevTools Protocol methods, and you can call them directly. This excerpt uses Playwright:

import { mkdtemp } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { chromium } from 'playwright';

// A persistent profile matters: browser.newContext() is off-the-record, and
// Chromium reports "in-incognito" as an installability error there.
const profile = await mkdtemp(path.join(tmpdir(), 'pwa-audit-'));
const context = await chromium.launchPersistentContext(profile, { serviceWorkers: 'allow' });
const page = context.pages()[0] ?? (await context.newPage());
await page.goto(process.argv[2], { waitUntil: 'load' });

const cdp = await context.newCDPSession(page);
// Manifest URL, raw JSON, parser errors (each with a `critical` flag) and resolved URLs.
const manifest = await cdp.send('Page.getAppManifest');
// Chromium's real installability check: [{ errorId, errorArguments }], e.g. "no-manifest".
const { installabilityErrors } = await cdp.send('Page.getInstallabilityErrors');

for (const e of manifest.errors ?? []) console.log(e.critical ? 'FAIL' : 'WARN', e.message);
for (const e of installabilityErrors) console.log('FAIL', e.errorId);
await context.close();
process.exitCode = installabilityErrors.length ? 1 : 0;

Both methods are experimental in the protocol, so pin your Playwright version. Passing this check means little on its own: Chromium's bar is now a name, a start URL, an app-like display mode and one icon of at least 144 px. Add the checks the old category never had: icon files compared against their declared sizes (a mislabeled icon passes Chromium's check), the worker's scope and Cache-Control headers, and an offline launch. For offline tests, use Playwright's context.setOffline(), which also applies to service workers; Puppeteer's page-level offline mode doesn't, so a network-first handler still reaches the network and the test passes for the wrong reason. Automated Testing has complete manifest, installability and offline test suites.

Keep Lighthouse for what it measures well (performance, accessibility and best practices) and run it twice: a cold run that matches a first visit and a warm run with the worker installed and controlling. If the warm run isn't clearly faster, your worker isn't helping navigations, usually because HTML is network-first without navigation preload or every request goes through a fetch handler that the Static Routing API could skip. The flow script, a maintained lighthouserc.cjs and a GitHub Actions workflow are in Lighthouse & Auditing.

Manual checks Chromium can't do for you¶

Every automated check above runs in Chromium. Safari and Firefox install web apps under different rules, and some of the most important PWA behavior only exists there. Before a release that touches install, the manifest or the worker, test these on real devices:

Record the results in the release checklist. The Production Checklist has a complete list.

Common pitfalls when replacing the category¶

• Treating "installable" as "done". An app can pass Page.getInstallabilityErrors with a manifest and one icon, and still show a browser error page when offline.
• Auditing in an incognito profile. Playwright's default contexts, and some CI browser setups, are off-the-record. You get in-incognito and nothing else.
• Reading a Lighthouse score as a PWA verdict. Default runs reset storage and never see your worker. A perfect score says nothing about offline support.
• Forgetting to test the real origin. CI on localhost skips HTTPS, HSTS, CDN headers and redirects. Run the checks against staging too.

Further reading¶

On this site

• Lighthouse & Auditing: the full workflow, custom audits, user flows and Lighthouse CI
• Browser DevTools: the Manifest and Service workers panes in depth
• Automated Testing: Playwright and Puppeteer patterns for PWAs
• Installability Criteria: every Chromium error ID and each browser's rules
• Production Checklist
• PWA Myths Debunked

External references

• Lighthouse 12.0.0 release notes, GoogleChrome/lighthouse
• Remove PWA Category (issue #15535), GoogleChrome/lighthouse
• Revisiting Chrome's installability criteria, Chrome for Developers
• Debug Progressive Web Apps, Chrome DevTools documentation
• Page domain: getInstallabilityErrors, Chrome DevTools Protocol
• Service workers, Playwright documentation
• Lighthouse CI, GoogleChrome/lighthouse-ci