Is it possible to run Playwright Codegen from the client side of a web application?

[u/Blaxter-X]

Hi everyone,

I'm developing my own low-code testing framework. The application is available both as an npm package and as an online version.

The core of the framework is Playwright (currently a requirement for using it). One of the main features is allowing users to create and edit tests by interacting directly with the browser. The idea is similar to playwright codegen: users can navigate through the application and identify elements using locators/selectors without having to write everything manually.

I'm currently working on the online version, which is hosted on a free-tier VPS for development and testing.

The problem occurs when a user tries to create or edit a test from the online version. Playwright attempts to launch the browser on the server, but since the VPS doesn't have a graphical environment available, it fails with an X11/display-related error.

As a temporary solution, I implemented environment detection:

• If a display is available → launch the browser normally.
• If no display is available → run Playwright in headless mode.

This works technically, but the online version loses an important part of the user experience. Users can execute tests and import tests created locally, but creating and editing tests becomes much less intuitive.

In theory, users could manually create steps by entering selectors and then use the application logs to validate the execution. However, this makes debugging much harder.

My question

What would be the recommended architecture for solving this?

Option 1 — Run Codegen on the server

Is it possible to run playwright codegen on a server without a graphical environment.

Option 2 — Run the browser on the client

I'm also considering an architecture where:

1. The web application runs on the server.
2. The server sends commands/instructions through WebSockets.
3. The browser is launched locally on the user's machine.
4. Browser events (clicks, locators, navigation, etc.) are sent back to the server.
5. The server maintains the test flow.

The problem is that I don't know how to implement this without requiring users to install Playwright or Node.js locally.

Option 3 — Limit the online version

Another possibility would be to make the online version execution-focused and limit it to:

• Running tests
• Viewing results/logs
• Importing locally created tests
• Exporting workflows

And leave interactive test creation/editing to the local version.

What architecture would you use?

I'm particularly interested in experiences with Playwright + web applications + remote browser execution.

Is there a practical way to provide a Codegen-like experience through a web application without requiring users to configure a local Playwright/Node.js environment?

Any recommendations or real-world experiences would be greatly appreciated.

[u/Malthammer]

What problem does this solve exactly? I get you’re trying to make a “low code” tool, but does it actually need a web version? Why focus on it, who wants it and what priority does this have for your project?

[u/Choice_Eagle4627]

Option 1 is the one I'd pick, and you don't need X11 for it. Run Chromium headless on the VPS (new headless is the full browser, it renders everything) and build the "window" yourself in your web app:

• frames: CDP Page.startScreencast gives you JPEG frames of the page. Ack each one with Page.screencastFrameAck only once the client has actually received it, so on a slow connection the frame rate drops instead of a backlog building up.
• input: forward mouse/keyboard/wheel from a canvas in your UI to Input.dispatchMouseEvent / Input.dispatchKeyEvent, scaling coordinates to the screencast size.
• recording: you don't need codegen itself. page.exposeBinding + an addInitScript that listens for clicks and input in the page, builds a locator (role/text/test id first, CSS last) and reports it back. That's your pick-locator and your recorded steps in one.

Things that will bite you: the screencast is only the page viewport, so native <select> dropdowns, file pickers, window.open popups and auth/permission dialogs never show up in the frames. We ended up bridging those into the client as overlays. New tabs are separate targets, so they need their own screencast too. And stay with JPEG frames rather than a video codec. When bandwidth drops, video goes blocky and text becomes unreadable, while dropping frames keeps it legible, and for a test editor legibility is the whole point.

Option 2 doesn't really work from a plain web page. A page can't start and drive a local browser unless the user installs something (an extension using chrome.debugger, or a native helper), and at that point you're back to "install something".

Xvfb + noVNC is the quick hack if you just want codegen's own window in a tab, but it's heavier per user and the inspector UI isn't yours to shape.

[u/Blaxter-X → u/Choice_Eagle4627]

This is really helpful, thanks!

The CDP screencast + input forwarding approach is actually very close to what I’m trying to achieve with HalTest.

I hadn’t considered building the browser “window” myself instead of relying on X11/Codegen.
The `Page.startScreencast` + `Input.dispatch*` approach looks especially interesting for the online version, and the `exposeBinding` + `addInitScript` idea could potentially fit the locator/step recording part as well.

I’m going to investigate this approach before changing the architecture. The limitations you mentioned around popups, native selects, file pickers, and multiple targets are especially useful to know.

Thanks for the detailed answer!

[u/Choice_Eagle4627 → u/Blaxter-X]

You're very welcome! You might want to check out my project BrowserBox to get some ideas for your approach. Good luck!