Skip to content

Evaluate Playwright CLI (@playwright/cli) as alternative to Playwright MCP Docker container #2444

@lpcox

Description

@lpcox

Problem

Playwright browser automation in AWF agentic workflows currently uses the Playwright MCP server (mcr.microsoft.com/playwright/mcp) running in a separate Docker container with --network host. This creates persistent networking challenges:

  1. Container-to-container routing: The agent container and Playwright container live on different networks, requiring host.docker.internal workarounds (see Playwright and local dev #2406, gh-aw#29825)
  2. Bridge IP fragility: Previous approach used bridge IP detection (ip -4 route get 1.1.1.1) which was unreliable across AWF configurations
  3. Extra container overhead: A full MCP server container for browser automation adds startup time, memory, and complexity
  4. Token inefficiency: MCP's large tool schemas and verbose accessibility trees consume model context

Proposal: Use @playwright/cli Instead

Microsoft now offers Playwright CLI (@playwright/cli), a CLI+SKILLS-based interface specifically designed for coding agents.

How It Works

# Install globally
npm install -g @playwright/cli@latest

# Install skills (for Claude Code, GitHub Copilot, etc.)
playwright-cli install --skills

# Direct usage
playwright-cli open http://localhost:4321/gh-aw/
playwright-cli screenshot
playwright-cli click "Get Started"
playwright-cli type "search query"

Why CLI is Better for AWF

From Microsoft's own documentation:

Modern coding agents increasingly favor CLI-based workflows exposed as SKILLs over MCP because CLI invocations are more token-efficient: they avoid loading large tool schemas and verbose accessibility trees into the model context, allowing agents to act through concise, purpose-built commands.

Factor MCP Container (current) CLI (proposed)
Networking ❌ Separate container, needs host.docker.internal ✅ Runs in agent container, native localhost access
Token usage ❌ Large tool schemas + accessibility trees in context ✅ Concise CLI commands, minimal context usage
Startup time ❌ Pull + start separate container ✅ Already installed, instant
Complexity ❌ MCP gateway routing, container orchestration ✅ Simple CLI calls via bash tool
localhost access ❌ Requires routing hacks ✅ Direct access to services in same container
Maintenance ❌ Container image pinning, digest updates ✅ npm dependency version

When MCP Is Still Appropriate

Per Microsoft's guidance, MCP remains relevant for:

  • Exploratory automation requiring persistent state
  • Self-healing tests with iterative reasoning over page structure
  • Long-running autonomous workflows needing continuous browser context

Most AWF smoke tests and doc testing workflows do not need these capabilities.

Implementation Plan

Phase 1: Proof of Concept

  1. Pre-install @playwright/cli in the AWF agent container image (or use npx @playwright/cli@latest)
  2. Run playwright-cli install --skills during container setup
  3. Test with a simple workflow: navigate, screenshot, assert

Phase 2: Migrate Existing Workflows

  • docs-noob-tester.md — currently uses bridge IP / host.docker.internal
  • daily-multi-device-docs-tester.md — viewport testing
  • visual-regression-checker.md — screenshot comparison
  • unbloat-docs.md — doc page analysis

Phase 3: AWF Integration

  • Add @playwright/cli as optional pre-installed tool in agent container
  • Update playwright_prompt.md in gh-aw setup actions
  • Consider a playwright-cli ecosystem identifier for network allowlist
  • Deprecate the mcr.microsoft.com/playwright/mcp container approach for standard use cases

Questions to Resolve

  1. Browser dependencies: Does @playwright/cli bundle Chromium, or does it need npx playwright install chromium?
  2. Headless support: Confirm headless-by-default works in AWF container (no display server)
  3. Screenshots: Where are screenshots saved? Can we control output directory?
  4. Skills integration: How do installed skills interact with different AI engines (Copilot, Claude, Codex)?
  5. Container size: How much does adding Playwright + Chromium increase the agent image size?

Related

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions