Opencli Browser

1---2name: opencli-browser3description: Make websites accessible for AI agents. Navigate, click, type, extract, wait — using Chrome with existing login sessions. No LLM API key needed.4allowed-tools: Bash(opencli:*), Read, Edit, Write5---6 7# OpenCLI Browser — Browser Automation for AI Agents8 9Control Chrome step-by-step via CLI. Reuses existing login sessions — no passwords needed.10 11## Prerequisites12 13```bash14opencli doctor    # Verify extension + daemon connectivity15```16 17Requires: Chrome running + OpenCLI Browser Bridge extension installed.18 19## Critical Rules20 211. **ALWAYS use `state` to inspect the page, NEVER use `screenshot`** — `state` returns structured DOM with `[N]` element indices, is instant and costs zero tokens. `screenshot` requires vision processing and is slow. Only use `screenshot` when the user explicitly asks to save a visual.222. **ALWAYS use `click`/`type`/`select` for interaction, NEVER use `eval` to click or type** — `eval "el.click()"` bypasses scrollIntoView and CDP click pipeline, causing failures on off-screen elements. Use `state` to find the `[N]` index, then `click <N>`.233. **Verify inputs with `get value`, not screenshots** — after `type`, run `get value <index>` to confirm.244. **Run `state` after every page change** — after `open`, `click` (on links), `scroll`, always run `state` to see the new elements and their indices. Never guess indices.255. **Chain commands aggressively with `&&`** — combine `open + state`, multiple `type` calls, and `type + get value` into single `&&` chains. Each tool call has overhead; chaining cuts it.266. **`eval` is read-only** — use `eval` ONLY for data extraction (`JSON.stringify(...)`), never for clicking, typing, or navigating. Always wrap in IIFE to avoid variable conflicts: `eval "(function(){ const x = ...; return JSON.stringify(x); })()"`.277. **Minimize total tool calls** — plan your sequence before acting. A good task completion uses 3-5 tool calls, not 15-20. Combine `open + state` as one call. Combine `type + type + click` as one call. Only run `state` separately when you need to discover new indices.288. **Prefer `network` to discover APIs** — most sites have JSON APIs. API-based adapters are more reliable than DOM scraping.29 30## Command Cost Guide31 32| Cost | Commands | When to use |33|------|----------|-------------|34| **Free & instant** | `state`, `get *`, `eval`, `network`, `scroll`, `keys` | Default — use these |35| **Free but changes page** | `open`, `click`, `type`, `select`, `back` | Interaction — run `state` after |36| **Expensive (vision tokens)** | `screenshot` | ONLY when user needs a saved image |37 38## Action Chaining Rules39 40Commands can be chained with `&&`. The browser persists via daemon, so chaining is safe.41 42**Always chain when possible** — fewer tool calls = faster completion:43```bash44# GOOD: open + inspect in one call (saves 1 round trip)45opencli browser open https://example.com && opencli browser state46 47# GOOD: fill form in one call (saves 2 round trips)48opencli browser type 3 "hello" && opencli browser type 4 "world" && opencli browser click 749 50# GOOD: type + verify in one call51opencli browser type 5 "test@example.com" && opencli browser get value 552 53# GOOD: click + wait + state in one call (for page-changing clicks)54opencli browser click 12 && opencli browser wait time 1 && opencli browser state55 56# BAD: separate calls for each action (wasteful)57opencli browser type 3 "hello"    # Don't do this58opencli browser type 4 "world"    # when you can chain59opencli browser click 7            # all three together60```61 62**Page-changing — always put last** in a chain (subsequent commands see stale indices):63- `open <url>`, `back`, `click <link/button that navigates>`64 65**Rule**: Chain when you already know the indices. Run `state` separately when you need to discover indices first.66 67## Core Workflow68 691. **Navigate**: `opencli browser open <url>`702. **Inspect**: `opencli browser state` → elements with `[N]` indices713. **Interact**: use indices — `click`, `type`, `select`, `keys`724. **Wait** (if needed): `opencli browser wait selector ".loaded"` or `wait text "Success"`735. **Verify**: `opencli browser state` or `opencli browser get value <N>`746. **Repeat**: browser stays open between commands757. **Save**: write a JS adapter to `~/.opencli/clis/<site>/<command>.js`76 77## Commands78 79### Navigation80 81```bash82opencli browser open <url>              # Open URL (page-changing)83opencli browser back                    # Go back (page-changing)84opencli browser scroll down             # Scroll (up/down, --amount N)85opencli browser scroll up --amount 100086```87 88### Inspect (free & instant)89 90```bash91opencli browser state                   # Structured DOM with [N] indices — PRIMARY tool92opencli browser screenshot [path.png]   # Save visual to file — ONLY for user deliverables93```94 95### Get (free & instant)96 97```bash98opencli browser get title               # Page title99opencli browser get url                 # Current URL100opencli browser get text <index>        # Element text content101opencli browser get value <index>       # Input/textarea value (use to verify after type)102opencli browser get html                # Full page HTML103opencli browser get html --selector "h1" # Scoped HTML104opencli browser get attributes <index>  # Element attributes105```106 107### Interact108 109```bash110opencli browser click <index>           # Click element [N]111opencli browser type <index> "text"     # Type into element [N]112opencli browser select <index> "option" # Select dropdown113opencli browser keys "Enter"            # Press key (Enter, Escape, Tab, Control+a)114```115 116### Wait117 118Three variants — use the right one for the situation:119 120```bash121opencli browser wait time 3                       # Wait N seconds (fixed delay)122opencli browser wait selector ".loaded"            # Wait until element appears in DOM123opencli browser wait selector ".spinner" --timeout 5000  # With timeout (default 30s)124opencli browser wait text "Success"                # Wait until text appears on page125```126 127**When to wait**: After `open` on SPAs, after `click` that triggers async loading, before `eval` on dynamically rendered content.128 129### Extract (free & instant, read-only)130 131Use `eval` ONLY for reading data. Never use it to click, type, or navigate.132 133```bash134opencli browser eval "document.title"135opencli browser eval "JSON.stringify([...document.querySelectorAll('h2')].map(e => e.textContent))"136 137# IMPORTANT: wrap complex logic in IIFE to avoid "already declared" errors138opencli browser eval "(function(){ const items = [...document.querySelectorAll('.item')]; return JSON.stringify(items.map(e => e.textContent)); })()"139```140 141**Selector safety**: Always use fallback selectors — `querySelector` returns `null` on miss:142```bash143# BAD: crashes if selector misses144opencli browser eval "document.querySelector('.title').textContent"145 146# GOOD: fallback with || or ?.147opencli browser eval "(document.querySelector('.title') || document.querySelector('h1') || {textContent:''}).textContent"148opencli browser eval "document.querySelector('.title')?.textContent ?? 'not found'"149```150 151### Network (API Discovery)152 153```bash154opencli browser network                  # Show captured API requests (auto-captured since open)155opencli browser network --detail 3       # Show full response body of request #3156opencli browser network --all            # Include static resources157```158 159### Sedimentation (Save as CLI)160 161```bash162opencli browser init hn/top              # Generate adapter scaffold at ~/.opencli/clis/hn/top.js163opencli browser verify hn/top            # Test the adapter (adds --limit 3 only if `limit` arg is defined)164```165 166- `init` auto-detects the domain from the active browser session (no need to specify it)167- `init` creates the file + populates `site`, `name`, `domain`, and `columns` from current page168- `verify` runs the adapter end-to-end and prints output; if no `limit` arg exists in the adapter, it won't pass `--limit 3`169 170### Session171 172```bash173opencli browser close                   # Close automation window174```175 176## Example: Extract HN Stories177 178```bash179opencli browser open https://news.ycombinator.com180opencli browser state                   # See [1] a "Story 1", [2] a "Story 2"...181opencli browser eval "JSON.stringify([...document.querySelectorAll('.titleline a')].slice(0,5).map(a => ({title: a.textContent, url: a.href})))"182opencli browser close183```184 185## Example: Fill a Form186 187```bash188opencli browser open https://httpbin.org/forms/post189opencli browser state                   # See [3] input "Customer Name", [4] input "Telephone"190opencli browser type 3 "OpenCLI" && opencli browser type 4 "555-0100"191opencli browser get value 3             # Verify: "OpenCLI"192opencli browser close193```194 195## Saving as Reusable CLI — Complete Workflow196 197### Step-by-step sedimentation flow:198 199```bash200# 1. Explore the website201opencli browser open https://news.ycombinator.com202opencli browser state                          # Understand DOM structure203 204# 2. Discover APIs (crucial for high-quality adapters)205opencli browser eval "fetch('/api/...').then(r=>r.json())"   # Trigger API calls206opencli browser network                        # See captured API requests207opencli browser network --detail 0             # Inspect response body208 209# 3. Generate scaffold210opencli browser init hn/top                    # Creates ~/.opencli/clis/hn/top.js211 212# 4. Edit the adapter (fill in func logic)213# - If API found: use fetch() directly (Strategy.PUBLIC or COOKIE)214# - If no API: use page.evaluate() for DOM extraction (Strategy.UI)215 216# 5. Verify217opencli browser verify hn/top                  # Runs the adapter and shows output218 219# 6. If verify fails, edit and retry220# 7. Close when done221opencli browser close222```223 224### Example adapter:225 226```typescript227// ~/.opencli/clis/hn/top.js228import { cli, Strategy } from '@jackwener/opencli/registry';229 230cli({231  site: 'hn',232  name: 'top',233  description: 'Top Hacker News stories',234  domain: 'news.ycombinator.com',235  strategy: Strategy.PUBLIC,236  browser: false,237  args: [{ name: 'limit', type: 'int', default: 5 }],238  columns: ['rank', 'title', 'score', 'url'],239  func: async (_page, kwargs) => {240    const limit = Math.min(Math.max(1, kwargs.limit ?? 5), 50);241    const resp = await fetch('https://hacker-news.firebaseio.com/v0/topstories.json');242    const ids = await resp.json();243    return Promise.all(244      ids.slice(0, limit).map(async (id: number, i: number) => {245        const item = await (await fetch(`https://hacker-news.firebaseio.com/v0/item/${id}.json`)).json();246        return { rank: i + 1, title: item.title, score: item.score, url: item.url ?? '' };247      })248    );249  },250});251```252 253Save to `~/.opencli/clis/<site>/<command>.js` → immediately available as `opencli <site> <command>`.254 255### Strategy Guide256 257| Strategy | When | browser: |258|----------|------|----------|259| `Strategy.PUBLIC` | Public API, no auth | `false` |260| `Strategy.COOKIE` | Needs login cookies | `true` |261| `Strategy.UI` | Direct DOM interaction | `true` |262 263**Always prefer API over UI** — if you discovered an API during browsing, use `fetch()` directly.264 265## Tips266 2671. **Always `state` first** — never guess element indices, always inspect first2682. **Sessions persist** — browser stays open between commands, no need to re-open2693. **Use `eval` for data extraction** — `eval "JSON.stringify(...)"` is faster than multiple `get` calls2704. **Use `network` to find APIs** — JSON APIs are more reliable than DOM scraping2715. **Alias**: `opencli op` is shorthand for `opencli browser`272 273## Common Pitfalls274 2751. **`form.submit()` fails in automation** — Don't use `form.submit()` or `eval` to submit forms. Navigate directly to the search URL instead:276   ```bash277   # BAD: form.submit() often silently fails278   opencli browser eval "document.querySelector('form').submit()"279   # GOOD: construct the URL and navigate280   opencli browser open "https://github.com/search?q=opencli&type=repositories"281   ```282 2832. **GitHub DOM changes frequently** — Prefer `data-testid` attributes when available; they are more stable than class names or tag structure.284 2853. **SPA pages need `wait` before extraction** — After `open` or `click` on single-page apps, the DOM isn't ready immediately. Always `wait selector` or `wait text` before `eval`.286 2874. **Use `state` before clicking** — Run `opencli browser state` to inspect available interactive elements and their indices. Never guess indices from memory.288 2895. **`evaluate` runs in browser context** — `page.evaluate()` in adapters executes inside the browser. Node.js APIs (`fs`, `path`, `process`) are NOT available. Use `fetch()` for network calls, DOM APIs for page data.290 2916. **Backticks in `page.evaluate` break JSON storage** — When writing adapters that will be stored/transported as JSON, avoid template literals inside `page.evaluate`. Use string concatenation or function-style evaluate:292   ```typescript293   // BAD: template literal backticks break when adapter is in JSON294   page.evaluate(`document.querySelector("${selector}")`)295   // GOOD: function-style evaluate296   page.evaluate((sel) => document.querySelector(sel), selector)297   ```298 299## Troubleshooting300 301| Error | Fix |302|-------|-----|303| "Browser not connected" | Run `opencli doctor` |304| "attach failed: chrome-extension://" | Disable 1Password temporarily |305| Element not found | `opencli browser scroll down && opencli browser state` |306| Stale indices after page change | Run `opencli browser state` again to get fresh indices |