feat: Phase 0 workspace, spikes, and golden harness

Stand up the monorepo skeleton and land the first Phase 0 artifacts:
pairing threat model, plugin catalog, builder detect, 20-prompt golden
harness, and synthetic 2GB mirror timing.
This commit is contained in:
SinachPat
2026-08-15 09:44:43 +01:00
parent 0dae13cfd3
commit 6e59a8a36b
59 changed files with 3440 additions and 109 deletions
+18
View File
@@ -0,0 +1,18 @@
# Copy to .env. Never commit real values.
# API
PORT=3000
DATABASE_URL=postgres://wursor:wursor@localhost:5432/wursor
REDIS_URL=redis://localhost:6379
# Auth
SESSION_SECRET=replace-me
# LLM — Grok is the default adapter. Switch provider with LLM_PROVIDER.
LLM_PROVIDER=grok
XAI_API_KEY=
LLM_FALLBACK_PROVIDER=
# Sandbox
DOCKER_HOST=
WARM_POOL_HOT_SPARES=2
View File
+6
View File
@@ -6,6 +6,12 @@ node_modules/
.wp-env/ .wp-env/
vendor/ vendor/
dist/ dist/
coverage/
playwright-report/
test-results/
*.tsbuildinfo
.phpunit.result.cache
*.log *.log
.idea/ .idea/
.vscode/ .vscode/
e2e/fixtures/large-exports/
+5 -5
View File
@@ -9,12 +9,12 @@ Non-technical WordPress site owners describe what they want; Wursor makes it hap
## Repo layout ## Repo layout
``` ```
api/ Node.js + TypeScript API server (session manager, agent orchestrator, api/ Node.js + TypeScript API server (empty until Phase 0 gate)
playbook runner, sandbox manager, deploy manager, plugin client) web/ React + TypeScript frontend (empty until Phase 0 gate)
web/ React + TypeScript frontend (chat, preview, approve/reject, deploy history) plugin/ WordPress plugin (PHP) — empty until Phase 0 gate
plugin/ WordPress plugin (PHP) — the connector on the user's hosting
infrastructure/ Docker images, warm pool, GC, deploy scripts infrastructure/ Docker images, warm pool, GC, deploy scripts
e2e/ Playwright end-to-end tests e2e/ Playwright + e2e/golden/ harness
spikes/ Phase 0 written results — gate before product code
PRD.md Product requirements (v2.0 — non-technical-first) PRD.md Product requirements (v2.0 — non-technical-first)
IMPLEMENTATION.md TDD build guide with 8-sprint Phase 1 plan IMPLEMENTATION.md TDD build guide with 8-sprint Phase 1 plan
``` ```
+258 -82
View File
@@ -11,7 +11,8 @@
1. [Architecture Overview](#1-architecture-overview) 1. [Architecture Overview](#1-architecture-overview)
2. [Project Structure](#2-project-structure) 2. [Project Structure](#2-project-structure)
3. [Build Phases](#3-build-phases) 3. [Build Phases](#3-build-phases)
4. [Phase 1Foundation (Weeks 18)](#4-phase-1--foundation-weeks-18) 4. [Phase 0Risk spikes (before Sprint 1)](#4-phase-0--risk-spikes-before-sprint-1)
5. [Phase 1 — Foundation (Weeks 18)](#5-phase-1--foundation-weeks-18)
- [Sprint 1: Web app scaffold + sandbox orchestration](#sprint-1) - [Sprint 1: Web app scaffold + sandbox orchestration](#sprint-1)
- [Sprint 2: WordPress plugin connector](#sprint-2) - [Sprint 2: WordPress plugin connector](#sprint-2)
- [Sprint 3: Agent orchestrator + playbook runner](#sprint-3) - [Sprint 3: Agent orchestrator + playbook runner](#sprint-3)
@@ -20,10 +21,10 @@
- [Sprint 6: Deploy + rollback](#sprint-6) - [Sprint 6: Deploy + rollback](#sprint-6)
- [Sprint 7: Integration + exit criteria](#sprint-7) - [Sprint 7: Integration + exit criteria](#sprint-7)
- [Sprint 8: Polish + alpha readiness](#sprint-8) - [Sprint 8: Polish + alpha readiness](#sprint-8)
5. [Phase 2 — Intelligence (Weeks 916)](#5-phase-2--intelligence-weeks-916) 6. [Phase 2 — Intelligence (Weeks 916)](#6-phase-2--intelligence-weeks-916)
6. [TDD Rules](#6-tdd-rules) 7. [TDD Rules](#7-tdd-rules)
7. [CI/CD Pipeline](#7-cicd-pipeline) 8. [CI/CD Pipeline](#8-cicd-pipeline)
8. [Glossary](#8-glossary) 9. [Glossary](#9-glossary)
--- ---
@@ -93,7 +94,7 @@
**Key stack decisions:** **Key stack decisions:**
- **Backend:** Node.js + TypeScript (fastest path to a working API server; the orchestration is I/O-bound, not CPU-bound) - **Backend:** Node.js + TypeScript (fastest path to a working API server; the orchestration is I/O-bound, not CPU-bound)
- **Frontend:** React + TypeScript (chat interface, preview iframe, deploy history) - **Frontend:** React + TypeScript (chat interface, preview iframe, deploy history)
- **Sandbox:** Docker containers on VPS with pre-baked WordPress image - **Sandbox:** Docker containers on VPS with a read-only pre-baked WordPress image and overlayfs site layers; media is proxied, not copied
- **Plugin:** PHP WordPress plugin (standard WordPress plugin architecture) - **Plugin:** PHP WordPress plugin (standard WordPress plugin architecture)
- **Database:** PostgreSQL for Wursor's own data (users, sites, sessions, deploy history); MySQL inside sandboxes for WordPress - **Database:** PostgreSQL for Wursor's own data (users, sites, sessions, deploy history); MySQL inside sandboxes for WordPress
- **Queue:** Redis for SSE streaming, task queues, and cache - **Queue:** Redis for SSE streaming, task queues, and cache
@@ -124,10 +125,12 @@ wursor/
│ │ │ ├── plugin-client.ts │ │ │ ├── plugin-client.ts
│ │ │ └── warm-pool.ts │ │ │ └── warm-pool.ts
│ │ ├── agents/ │ │ ├── agents/
│ │ │ ├── grok-client.ts # Grok API client │ │ │ ├── llm-client.ts # Provider-agnostic LLM client (Grok adapter default)
│ │ │ ├── prompt-builder.ts # System prompt per session │ │ │ ├── grok-adapter.ts # Grok messages + tool-calling
│ │ │ ├── tool-schemas.ts # Tool schemas → Grok format │ │ │ ├── prompt-builder.ts # System prompt per session (playbook-sliced)
│ │ │ ── fallback.ts # Error handling, retry │ │ │ ── tool-schemas.ts # Allowlisted tool schemas only
│ │ │ ├── circuit-breaker.ts # Two verify failures → stop
│ │ │ └── fallback.ts # Per-playbook fallback + retry
│ │ ├── playbooks/ │ │ ├── playbooks/
│ │ │ ├── registry.ts # Playbook registry │ │ │ ├── registry.ts # Playbook registry
│ │ │ ├── content.ts # Content edit playbook │ │ │ ├── content.ts # Content edit playbook
@@ -135,16 +138,20 @@ wursor/
│ │ │ ├── plugin.ts # Plugin install playbook │ │ │ ├── plugin.ts # Plugin install playbook
│ │ │ └── site-build.ts # Site build playbook (P0 limited) │ │ │ └── site-build.ts # Site build playbook (P0 limited)
│ │ ├── sandbox/ │ │ ├── sandbox/
│ │ │ ├── docker-client.ts # Docker API client │ │ │ ├── docker-client.ts # Docker API client (overlayfs + pause)
│ │ │ ├── image-manager.ts # Pre-baked image management │ │ │ ├── image-manager.ts # Pre-baked image management
│ │ │ ├── mirror.ts # Site mirroring (content, themes, plugins) │ │ │ ├── mirror.ts # Task-scoped site mirroring
│ │ │ ├── media-sync.ts # Lazy media sync │ │ │ ├── media-proxy.ts # Origin proxy for /wp-content/uploads
│ │ │ ── gc.ts # Garbage collection (idle, hard timeout) │ │ │ ── subset.ts # DB subset + secret redaction
│ │ │ ├── manifest.ts # path → sha256 delta + package cache
│ │ │ └── gc.ts # Pause-to-disk, idle + hard timeout
│ │ ├── deploy/ │ │ ├── deploy/
│ │ │ ├── diff-engine.ts # Compare sandbox → live site │ │ │ ├── diff-engine.ts # Compare sandbox → live site
│ │ │ ├── pusher.ts # Push changes via plugin API │ │ │ ├── pusher.ts # Two-phase prepare/commit + journal
│ │ │ ├── verifier.ts # Verify live site after deploy │ │ │ ├── verifier.ts # Health contract (sandbox + live)
│ │ │ ── rollback.ts # Snapshot-based rollback │ │ │ ── drift.ts # Re-hash live site at approve time
│ │ │ ├── no-surprise.ts # Block slug/payment/role without confirm
│ │ │ └── rollback.ts # Journal walk + cloud snapshot restore
│ │ ├── models/ │ │ ├── models/
│ │ │ ├── user.ts │ │ │ ├── user.ts
│ │ │ ├── site.ts │ │ │ ├── site.ts
@@ -162,20 +169,24 @@ wursor/
│ │ │ ├── deploy-manager.test.ts │ │ │ ├── deploy-manager.test.ts
│ │ │ └── playbook-runner.test.ts │ │ │ └── playbook-runner.test.ts
│ │ ├── agents/ │ │ ├── agents/
│ │ │ ├── grok-client.test.ts │ │ │ ├── llm-client.test.ts
│ │ │ ├── prompt-builder.test.ts │ │ │ ├── prompt-builder.test.ts
│ │ │ ── tool-schemas.test.ts │ │ │ ── tool-schemas.test.ts
│ │ │ └── circuit-breaker.test.ts
│ │ ├── playbooks/ │ │ ├── playbooks/
│ │ │ ├── content.test.ts │ │ │ ├── content.test.ts
│ │ │ ├── design.test.ts │ │ │ ├── design.test.ts
│ │ │ └── plugin.test.ts │ │ │ └── plugin.test.ts
│ │ ├── sandbox/ │ │ ├── sandbox/
│ │ │ ├── mirror.test.ts │ │ │ ├── mirror.test.ts
│ │ │ ├── media-sync.test.ts │ │ │ ├── media-proxy.test.ts
│ │ │ ├── subset.test.ts
│ │ │ └── gc.test.ts │ │ │ └── gc.test.ts
│ │ └── deploy/ │ │ └── deploy/
│ │ ├── diff-engine.test.ts │ │ ├── diff-engine.test.ts
│ │ ├── pusher.test.ts │ │ ├── pusher.test.ts
│ │ ├── drift.test.ts
│ │ ├── no-surprise.test.ts
│ │ └── rollback.test.ts │ │ └── rollback.test.ts
│ ├── package.json │ ├── package.json
│ └── tsconfig.json │ └── tsconfig.json
@@ -268,9 +279,24 @@ wursor/
--- ---
## 4. Phase 1Foundation (Weeks 18) ## 4. Phase 0Risk spikes (before Sprint 1)
8 sprints, one per week. Every sprint produces a passing integration test. These are not optional research notes. They lock decisions that Sprints 13 will encode as tests. Do not start the web-app scaffold until the four boxes below have a written result in this repo (a `spikes/` note is enough).
| Spike | Question | Done when |
| :--- | :--- | :--- |
| **Golden-task harness (R7)** | Can we score a model on WordPress tasks without vibes? | 20 fixture prompts against at least 2 canned WP sites. Each prompt has an assertion (preview text, option value, or screenshot). One Grok run is scored. Harness lives under `e2e/golden/` even if the runner is still a script. |
| **Elementor detect (R6 / R13)** | How do we know what actually renders a page? | A site-info payload that reports `builder: elementor \| beaver \| divi \| gutenberg \| classic` from plugin slugs + post meta. Documented in the plugin API sketch. |
| **Pairing threat model (R9)** | What stops a leaked URL from owning the site? | Written threat model: 8+ char code, 5-min TTL, 5-attempt lockout, HMAC request signing, hashed+scoped tokens. This becomes the Sprint 2 auth tests. |
| **Large-site mirror timing (R4)** | Does the 5-minute exit criterion survive a real site? | Time a task-scoped content mirror + media proxy against one ≥2GB WP export (or a synthetic one). Record p50/p95. If content-edit slice is not on the page in ≤60s, the Sprint 1 slice is wrong. |
Also decide, in writing, the P0 plugin catalog (~40 slugs). The agent will not be allowed to install anything else.
---
## 5. Phase 1 — Foundation (Weeks 18)
8 sprints, one per week. Every sprint produces a passing integration test. Risk IDs refer to [PRD.md §13](./PRD.md).
--- ---
@@ -326,8 +352,24 @@ describe('Mirror', () => {
expect(await mirror.sandboxFileExists('/wp-content/themes/twentytwentyfour')).toBe(true); expect(await mirror.sandboxFileExists('/wp-content/themes/twentytwentyfour')).toBe(true);
}); });
it('lazy-syncs media only when accessed', async () => { it('does not copy the media library; preview is served via origin proxy', async () => {
// Media should not be synced during mirror, only on first access const mirror = new Mirror({ sandboxId: 'sb-123' });
await mirror.copyTheme('twentytwentyfour');
expect(await mirror.sandboxFileExists('/wp-content/uploads/2024/hero.jpg')).toBe(false);
expect(await mirror.mediaProxyTarget('/wp-content/uploads/2024/hero.jpg')).toMatch(/^https:\/\/example.com\//);
});
it('copies a media file only when the agent replaces it', async () => {
const mirror = new Mirror({ sandboxId: 'sb-123' });
await mirror.stageReplacement('/wp-content/uploads/2024/hero.jpg', Buffer.from('new'));
expect(await mirror.sandboxFileExists('/wp-content/uploads/2024/hero.jpg')).toBe(true);
});
it('mirrors a content-edit slice, not orders or transients', async () => {
const dump = await new Mirror({ sandboxId: 'sb-123' }).exportDbSubset({ playbook: 'content', postIds: [1] });
expect(dump.tables).toEqual(expect.arrayContaining(['wp_posts', 'wp_postmeta', 'wp_options']));
expect(dump.tables).not.toEqual(expect.arrayContaining(['wp_wc_orders', 'wp_comments']));
expect(dump.options).not.toEqual(expect.arrayContaining([expect.stringMatching(/(_key|_secret|smtp_pass)$/)]));
}); });
}); });
@@ -350,9 +392,10 @@ describe('DockerClient', () => {
// api/__tests__/sandbox/gc.test.ts // api/__tests__/sandbox/gc.test.ts
describe('GarbageCollection', () => { describe('GarbageCollection', () => {
it('destroys sandboxes after 15 minutes of idle', async () => { /* ... */ }); it('pauses sandboxes to disk after 15 minutes of idle', async () => { /* ... */ });
it('resumes a paused sandbox in ≤ 2s', async () => { /* ... */ });
it('destroys sandboxes after 24 hours regardless', async () => { /* ... */ }); it('destroys sandboxes after 24 hours regardless', async () => { /* ... */ });
it('does not destroy active sandboxes', async () => { /* ... */ }); it('does not pause or destroy active sandboxes', async () => { /* ... */ });
}); });
``` ```
@@ -365,16 +408,20 @@ describe('GarbageCollection', () => {
- **`api/src/services/sandbox-manager.ts`** — Orchestrate sandbox lifecycle - **`api/src/services/sandbox-manager.ts`** — Orchestrate sandbox lifecycle
- **`api/src/sandbox/docker-client.ts`** — Docker API client (dockerode) - **`api/src/sandbox/docker-client.ts`** — Docker API client (dockerode)
- **`api/src/sandbox/image-manager.ts`** — Pre-baked image → Dockerfile - **`api/src/sandbox/image-manager.ts`** — Pre-baked image → Dockerfile
- **`api/src/sandbox/mirror.ts`** — Site mirroring (stub plugin client) - **`api/src/sandbox/mirror.ts`** — Task-scoped site mirroring (stub plugin client)
- **`api/src/sandbox/media-sync.ts`** — Lazy media sync (stub) - **`api/src/sandbox/media-proxy.ts`** — Origin rewrite for `/wp-content/uploads` (no library copy)
- **`api/src/sandbox/gc.ts`** — Garbage collection (idle timeout, hard timeout) - **`api/src/sandbox/subset.ts`** — DB subset + `*_key` / `*_secret` / `smtp_pass` redaction (R10)
- **`infrastructure/docker/Dockerfile.wordpress`** — Pre-baked image - **`api/src/sandbox/manifest.ts`** — path→sha256 delta; wordpress.org packages from Wursor cache
- **`infrastructure/scripts/warm-pool.ts`** — Warm pool manager - **`api/src/sandbox/gc.ts`** — Pause-to-disk on idle; destroy on 24h hard timeout
- **`infrastructure/docker/Dockerfile.wordpress`** — Pre-baked image (read-only base + overlayfs)
- **`infrastructure/scripts/warm-pool.ts`** — Paused images + 12 hot spares, not 510 running
#### Deliverables #### Deliverables
- Web app with sign-up and chat interface - Web app with sign-up and chat interface
- Sandbox spin-up from pre-baked image - Sandbox spin-up from pre-baked image (overlay + pause)
- Media proxy: preview works with zero upload copy
- Content-edit DB subset excludes orders / secrets
- `e2e/chat-flow.test.ts` passing (sign-up → sees chat) - `e2e/chat-flow.test.ts` passing (sign-up → sees chat)
- All unit tests passing - All unit tests passing
@@ -394,8 +441,24 @@ class WursorAuthTest extends WP_UnitTestCase {
public function test_generates_pairing_code() { public function test_generates_pairing_code() {
$auth = new Wursor_Auth(); $auth = new Wursor_Auth();
$code = $auth->generate_pairing_code(); $code = $auth->generate_pairing_code();
$this->assertEquals(6, strlen($code)); $this->assertGreaterThanOrEqual(8, strlen($code));
$this->assertMatchesRegularExpression('/^[A-Z0-9]{6}$/', $code); $this->assertMatchesRegularExpression('/^[A-Z0-9]{8,}$/', $code);
}
public function test_pairing_code_expires_after_five_minutes() {
$auth = new Wursor_Auth();
$code = $auth->generate_pairing_code();
$auth->advance_clock(301);
$this->assertFalse($auth->redeem_pairing_code($code));
}
public function test_locks_out_after_five_failed_attempts() {
$auth = new Wursor_Auth();
$auth->generate_pairing_code();
for ($i = 0; $i < 5; $i++) {
$auth->redeem_pairing_code('NOPE0000');
}
$this->assertTrue($auth->is_locked_out());
} }
public function test_verifies_valid_token() { public function test_verifies_valid_token() {
@@ -419,6 +482,9 @@ class WursorApiTest extends WP_UnitTestCase {
$this->assertArrayHasKey('plugins', $response); $this->assertArrayHasKey('plugins', $response);
$this->assertArrayHasKey('wordpress_version', $response); $this->assertArrayHasKey('wordpress_version', $response);
$this->assertArrayHasKey('php_version', $response); $this->assertArrayHasKey('php_version', $response);
$this->assertArrayHasKey('builder', $response);
$this->assertArrayHasKey('capabilities', $response);
$this->assertArrayHasKey('preflight', $response);
} }
public function test_requires_auth() { public function test_requires_auth() {
@@ -472,19 +538,21 @@ describe('SiteConnector', () => {
**Step 2 — Implement** **Step 2 — Implement**
- **`plugin/wursor.php`** — Plugin header, activation hook, bootstrap - **`plugin/wursor.php`** — Plugin header, activation hook, bootstrap
- **`plugin/src/class-auth.php`** — Token generation, verification, pairing code - **`plugin/src/class-auth.php`** — 8+ char pairing (5-min TTL, 5-attempt lockout), hashed scoped tokens (read vs deploy), HMAC request signing (R9)
- **`plugin/src/class-api.php`** — REST API endpoints (site-info, files, DB, WP-CLI) - **`plugin/src/class-api.php`** — REST API endpoints (site-info, files, DB, WP-CLI); HMAC verified
- **`plugin/src/class-site-info.php`** — Site info provider (theme, plugins, WP version, PHP version) - **`plugin/src/class-site-info.php`** — Theme, plugins, WP/PHP version, `builder`, capability tiers, pre-flight probe (disk, `DISALLOW_FILE_MODS`, cache flush, REST alive)
- **`plugin/src/class-admin.php`** — Admin settings page (pairing code display) - **`plugin/src/class-admin.php`** — Admin settings page (pairing code display)
- **`api/src/services/plugin-client.ts`** — HTTP client for the plugin API - **`api/src/services/plugin-client.ts`** — HTTP client for the plugin API (signs requests)
- **`api/src/routes/sites.ts`** — Site connection flow, pairing - **`api/src/routes/sites.ts`** — Site connection flow, pairing, capability-tier response
- **`web/src/components/SiteConnector.tsx`** — Pairing UI (show code, wait for connection) - **`web/src/components/SiteConnector.tsx`** — Pairing UI (show code, wait for connection)
- **`web/src/pages/ConnectSite.tsx`** — Connection page - **`web/src/pages/ConnectSite.tsx`** — Connection page; tiered copy (“I can change text today…”) + concierge host-ticket email when PHP/WP is below the full-playbook matrix
#### Deliverables #### Deliverables
- WordPress plugin with pairing and site-info API - WordPress plugin with pairing and site-info API
- Plugin client in the API server - Pairing is 8+ chars, TTL + lockout tested; tokens scoped and HMAC-signed
- `builder` + `capabilities` + `preflight` on site-info
- Content-only tier for WP 5.86.0 / PHP 7.4; full playbooks for WP 6.1+ / PHP 8.0+
- Connection flow: user installs plugin → gets code → enters in Wursor → connected - Connection flow: user installs plugin → gets code → enters in Wursor → connected
- `plugin/__tests__/test-auth.php` and `test-api.php` passing - `plugin/__tests__/test-auth.php` and `test-api.php` passing
- `api/__tests__/services/plugin-client.test.ts` passing - `api/__tests__/services/plugin-client.test.ts` passing
@@ -501,16 +569,21 @@ describe('SiteConnector', () => {
**Step 1 — Write the tests** **Step 1 — Write the tests**
```typescript ```typescript
// api/__tests__/agents/grok-client.test.ts // api/__tests__/agents/llm-client.test.ts
describe('GrokClient', () => { describe('LlmClient', () => {
it('sends a message and returns a response', async () => { it('sends a message and returns a response via the Grok adapter', async () => {
const client = new GrokClient({ apiKey: 'test-key' }); const client = new LlmClient({ provider: 'grok', apiKey: 'test-key' });
const response = await client.send('Change the homepage heading to "Hello"'); const response = await client.send('Change the homepage heading to "Hello"');
expect(response.type).toBe('tool_call'); expect(response.type).toBe('tool_call');
}); });
it('switches provider with an env/config change, not a rewrite', async () => {
const client = new LlmClient({ provider: 'fallback', apiKey: 'test-key' });
expect(client.provider).toBe('fallback');
});
it('handles API errors with a clear message', async () => { it('handles API errors with a clear message', async () => {
const client = new GrokClient({ apiKey: 'invalid-key' }); const client = new LlmClient({ provider: 'grok', apiKey: 'invalid-key' });
await expect(client.send('hello')).rejects.toThrow('API error'); await expect(client.send('hello')).rejects.toThrow('API error');
}); });
@@ -540,12 +613,33 @@ describe('PromptBuilder', () => {
// api/__tests__/agents/tool-schemas.test.ts // api/__tests__/agents/tool-schemas.test.ts
describe('ToolSchemas', () => { describe('ToolSchemas', () => {
it('generates tool schemas for the Grok API', () => { it('generates tool schemas for the LLM provider', () => {
const schemas = generateToolSchemas(); const schemas = generateToolSchemas();
expect(schemas.length).toBeGreaterThan(0); expect(schemas.length).toBeGreaterThan(0);
expect(schemas[0].name).toBe('wp_cli'); expect(schemas[0].name).toBe('wp_cli');
expect(schemas[0].parameters).toBeDefined(); expect(schemas[0].parameters).toBeDefined();
}); });
it('does not expose eval, config, db DROP, rm, or arbitrary plugin URLs', () => {
const names = generateToolSchemas().flatMap((s) => [s.name, ...(s.parameters?.enum ?? [])]);
expect(names.join(' ')).not.toMatch(/wp eval|wp config|DROP TABLE|wp plugin install http/);
});
});
// api/__tests__/agents/circuit-breaker.test.ts
describe('CircuitBreaker', () => {
it('stops the agent after two consecutive verify failures', async () => {
const breaker = new CircuitBreaker({ maxConsecutiveFailures: 2 });
await breaker.recordFailure();
await breaker.recordFailure();
expect(breaker.shouldHalt()).toBe(true);
});
it('halts when the per-task token budget is exhausted', async () => {
const breaker = new CircuitBreaker({ maxToolRounds: 12, maxUsd: 0.5 });
breaker.recordUsage({ rounds: 12, usd: 0.1 });
expect(breaker.shouldHalt()).toBe(true);
});
}); });
// api/__tests__/services/agent-orchestrator.test.ts // api/__tests__/services/agent-orchestrator.test.ts
@@ -594,12 +688,14 @@ describe('ChatPanel', () => {
**Step 2 — Implement** **Step 2 — Implement**
- **`api/src/agents/grok-client.ts`** — Grok API client (messages API, tool use, streaming) - **`api/src/agents/llm-client.ts`** — Provider-agnostic client; Grok is the default adapter
- **`api/src/agents/prompt-builder.ts`** — Build system prompt per session - **`api/src/agents/grok-adapter.ts`** — Grok messages API, tool use, streaming
- **`api/src/agents/tool-schemas.ts`** — Tool schemas → Grok format - **`api/src/agents/prompt-builder.ts`** — Playbook-sliced system prompt (do not dump the whole site-info blob)
- **`api/src/agents/fallback.ts`** — Error handling, retry - **`api/src/agents/tool-schemas.ts`** — Allowlisted tools only (R2, R7)
- **`api/src/services/agent-orchestrator.ts`** — Route requests, dispatch tools, stream results - **`api/src/agents/circuit-breaker.ts`** — Two verify failures or budget cap → halt (R1, R8)
- **`api/src/services/playbook-runner.ts`** — Execute playbook steps in sandbox - **`api/src/agents/fallback.ts`** — Per-playbook fallback provider + retry
- **`api/src/services/agent-orchestrator.ts`** — Route requests, dispatch tools, stream results, enforce budget
- **`api/src/services/playbook-runner.ts`** — Execute playbook steps in sandbox; checkpoint after each success
- **`api/src/playbooks/registry.ts`** — Playbook registry - **`api/src/playbooks/registry.ts`** — Playbook registry
- **`api/src/routes/chat.ts`** — Chat message endpoint, SSE stream - **`api/src/routes/chat.ts`** — Chat message endpoint, SSE stream
- **`web/src/components/ChatPanel.tsx`** — Chat UI with message list, input, typing indicator - **`web/src/components/ChatPanel.tsx`** — Chat UI with message list, input, typing indicator
@@ -608,7 +704,9 @@ describe('ChatPanel', () => {
#### Deliverables #### Deliverables
- Agent orchestrator routing requests to playbooks - Agent orchestrator routing requests to playbooks
- Grok API client with tool-calling - Provider-agnostic LLM client with Grok adapter and per-playbook fallback
- Tool allowlist: no `wp eval`, `wp config`, DROP, `rm`, or `wp plugin install <url>`
- Circuit breaker + per-task token/round budget
- Chat panel streaming agent responses - Chat panel streaming agent responses
- All unit tests passing - All unit tests passing
@@ -670,8 +768,15 @@ describe('ContentPlaybook', () => {
expect(result.success).toBe(true); expect(result.success).toBe(true);
}); });
it('reverts changes on failure', async () => { it('rewinds the last checkpoint when a change fails verify, instead of nuking the sandbox', async () => {
// If the change fails, the sandbox should be reset to the mirror state const playbook = new ContentPlaybook({ sandboxId: 'sb-123' });
await playbook.editText({ page: 'homepage', target: 'Hello', replacement: 'World' });
const checkpoint = await playbook.lastCheckpoint();
await playbook.editText({ page: 'homepage', target: 'World', replacement: '<broken' });
expect(await playbook.verify()).toMatchObject({ ok: false });
await playbook.rewind();
expect(await playbook.currentCheckpoint()).toBe(checkpoint);
expect(await playbook.getPageContent('homepage')).toContain('World');
}); });
}); });
``` ```
@@ -679,11 +784,12 @@ describe('ContentPlaybook', () => {
**Step 2 — Implement** **Step 2 — Implement**
- **`api/src/playbooks/content.ts`** — Content playbook with tool calls - **`api/src/playbooks/content.ts`** — Content playbook with tool calls
- `editText`: search DB for content → wp-cli `wp post update` or direct DB update - `editText`: search DB for content → `wp post update` or REST. **No theme PHP edits for copy changes (R1).**
- `editHeading`: find heading in page HTML → update via WP-CLI or file edit - `editHeading`: find heading via builder adapter (Gutenberg / Elementor / Classic) → update via REST or that builder's store
- `replaceImage`: upload new image → replace in content → verify - `replaceImage`: upload new image → replace in content → verify (this is the one case that copies a media file into the sandbox)
- `addSection`: create new content block → add to page → verify - `addSection`: create new content block via the detected builder adapter → add to page → verify
- Each method uses the plugin client to execute WP-CLI commands or file operations in the sandbox - Each successful step writes a copy-on-write checkpoint
- Verify (health contract) runs after every step; failure rewinds one checkpoint
- After each change, the playbook triggers a preview refresh - After each change, the playbook triggers a preview refresh
#### Deliverables #### Deliverables
@@ -749,10 +855,11 @@ describe('DesignPlaybook', () => {
- **`api/src/playbooks/design.ts`** — Design playbook - **`api/src/playbooks/design.ts`** — Design playbook
- `changeTheme`: install theme via WP-CLI → activate → verify - `changeTheme`: install theme via WP-CLI → activate → verify
- `changeLayout`: modify theme templates or page builder content → verify - `changeLayout`: modify theme templates **or the detected builder adapter** (Elementor JSON / Gutenberg blocks / Classic HTML) → verify. Editing the wrong store is a failed test (R6, R13).
- `updateColors`: update theme.json → regenerate CSS → verify - `updateColors`: update theme.json → regenerate CSS → verify
- `updateTypography`: update theme.json → verify - `updateTypography`: update theme.json → verify
- `fixMobileLayout`: identify responsive CSS issues → fix → verify - `fixMobileLayout`: identify responsive CSS issues → fix → verify
- Deploy lints any written PHP against the live site's declared PHP version
#### Deliverables #### Deliverables
@@ -817,8 +924,26 @@ describe('Pusher', () => {
expect(result.success).toBe(true); expect(result.success).toBe(true);
}); });
it('handles partial failures', async () => { it('prepare-fails without touching the live site', async () => {
// If some files fail but others succeed, what happens? Roll back the batch. const pusher = new Pusher({ siteUrl: 'https://example.com', token: 'valid-token' });
pusher.failNextPrepare();
const result = await pusher.push({ files: [{ path: '/wp-content/themes/twentytwentyfour/style.css', content: '...' }] });
expect(result.success).toBe(false);
expect(result.liveTouched).toBe(false);
});
it('rolls back the journal when commit is partial', async () => {
const pusher = new Pusher({ siteUrl: 'https://example.com', token: 'valid-token' });
pusher.failOnJournalEntry(2);
const result = await pusher.push({
files: [
{ path: '/wp-content/themes/twentytwentyfour/style.css', content: 'a' },
{ path: '/wp-content/themes/twentytwentyfour/theme.json', content: 'b' },
],
});
expect(result.success).toBe(false);
expect(result.rolledBack).toBe(true);
expect(await pusher.liveFile('/wp-content/themes/twentytwentyfour/style.css')).not.toBe('a');
}); });
}); });
@@ -841,6 +966,39 @@ describe('Verifier', () => {
const result = await verifier.checkAdmin(); const result = await verifier.checkAdmin();
expect(result.status).toBe(200); expect(result.status).toBe(200);
}); });
it('fails when siteurl/home drifted from intent', async () => {
const verifier = new Verifier({ siteUrl: 'https://example.com', intent: { home: 'https://example.com' } });
await verifier.injectOption('home', 'https://evil.example');
const result = await verifier.checkIntent();
expect(result.ok).toBe(false);
});
it('fails a 200 homepage whose screenshot diverges from the sandbox', async () => {
const verifier = new Verifier({ siteUrl: 'https://example.com', sandboxUrl: 'http://sb-123.wursor.dev' });
const result = await verifier.checkScreenshotSsim();
expect(result.ssim).toBeGreaterThan(0.9);
});
});
// api/__tests__/deploy/drift.test.ts
describe('Drift', () => {
it('asks before deploy when the live site changed since the preview', async () => {
const drift = new DriftChecker({ siteUrl: 'https://example.com', token: 'valid-token' });
const verdict = await drift.compare(changesetTakenAtPreview);
expect(verdict.drifted).toBe(true);
expect(verdict.action).toBe('confirm');
});
});
// api/__tests__/deploy/no-surprise.test.ts
describe('NoSurprise', () => {
it('blocks slug, blog_public, payment, and role changes without an explicit confirm bullet', async () => {
const gate = new NoSurpriseGate();
const blocked = gate.review({ changedOptions: ['permalink_structure'], confirmed: [] });
expect(blocked.ok).toBe(false);
expect(blocked.bullets).toContain('permalink_structure');
});
}); });
// api/__tests__/deploy/rollback.test.ts // api/__tests__/deploy/rollback.test.ts
@@ -906,22 +1064,28 @@ describe('ApproveBar', () => {
**Step 2 — Implement** **Step 2 — Implement**
- **`plugin/src/class-deploy.php`** — Deploy receiver (file write, DB write, WP-CLI exec, snapshot) - **`plugin/src/class-deploy.php`** — Two-phase prepare/commit, journaled file/DB/WP-CLI, maintenance mode for the commit window
- **`plugin/src/class-rollback.php`** — Snapshot-based rollback (files + DB) - **`plugin/src/class-rollback.php`** — Journal walk-back + snapshot restore
- **`api/src/deploy/diff-engine.ts`** — Compare sandbox → live site - **`api/src/deploy/diff-engine.ts`** — Compare sandbox → live site
- **`api/src/deploy/pusher.ts`** — Push changes via plugin API - **`api/src/deploy/pusher.ts`** — Prepare then commit; never leave a partial live site
- **`api/src/deploy/verifier.ts`** — Verify live site after deploy - **`api/src/deploy/verifier.ts`** — Health contract on sandbox *and* live (R1, R3)
- **`api/src/deploy/rollback.ts`** — Snapshot-based rollback - **`api/src/deploy/drift.ts`** — Re-hash live site at approve time (R11)
- **`api/src/routes/deploy.ts`** — Approve, deploy, rollback endpoints - **`api/src/deploy/no-surprise.ts`** — Block slug / `blog_public` / payment / role without confirm (R12)
- **`web/src/components/ApproveBar.tsx`** — Approve/reject buttons, confirmation dialog - **`api/src/deploy/rollback.ts`** — Journal + last-3 cloud snapshots (Undo works if the site is down)
- **`api/src/routes/deploy.ts`** — Approve, deploy, rollback; first-N-deploys “watched” path (R14)
- **`web/src/components/ApproveBar.tsx`** — Approve/reject, confirmation, no-surprise bullets, drift prompt
- **`web/src/components/DeployTimeline.tsx`** — Deploy history with one-click undo - **`web/src/components/DeployTimeline.tsx`** — Deploy history with one-click undo
- **`web/src/hooks/useDeploy.ts`** — Deploy state, polling - **`web/src/hooks/useDeploy.ts`** — Deploy state, polling
#### Deliverables #### Deliverables
- Deploy + rollback for files, DB, and plugins - Two-phase deploy + journaled rollback for files, DB, and plugins
- Cloud copies of the last 3 snapshots
- Drift check and no-surprise gate on approve
- Health contract richer than HTTP 200
- Approve/reject UI with confirmation dialog - Approve/reject UI with confirmation dialog
- Deploy history timeline with one-click undo - Deploy history timeline with one-click undo
- `handles partial failures` is a real test, not a comment
- All unit tests passing - All unit tests passing
--- ---
@@ -948,7 +1112,7 @@ test('new user connects site, makes a content change, previews, approves in ≤5
// 2. Connect site (simulated plugin) // 2. Connect site (simulated plugin)
await expect(page.locator('.wursor-connect-site')).toBeVisible(); await expect(page.locator('.wursor-connect-site')).toBeVisible();
await page.locator('.wursor-pairing-code-input').fill('ABC123'); await page.locator('.wursor-pairing-code-input').fill('ABCD1234');
await page.locator('.wursor-connect-button').click(); await page.locator('.wursor-connect-button').click();
await expect(page.locator('.wursor-connected')).toBeVisible({ timeout: 10000 }); await expect(page.locator('.wursor-connected')).toBeVisible({ timeout: 10000 });
@@ -984,39 +1148,47 @@ test('new user connects site, makes a content change, previews, approves in ≤5
#### Tasks #### Tasks
- **Error states** — Wire each state from §8.5 into the UI - **Error states** — Wire each state from §8.5 into the UI
- **Intent chips (R5)** — Empty state: “Change wording” / “New look” / “Add a form” / “Somethings broken”
- **Site-aware starters (R5)** — After connect, offer three specific sentences from the site scan (default H1, missing favicon, no contact page)
- **Structured reject (R5)** — Chips: “Wrong color” / “Too busy” / “Keep my logo” / “Undo only the last thing”
- **Point-and-talk spike** — Click in the preview → selector attached to the next chat turn
- **Mobile responsive** — Chat collapses to full-screen on mobile, preview opens in new tab - **Mobile responsive** — Chat collapses to full-screen on mobile, preview opens in new tab
- **Email auth** — Magic link or password reset flow - **Email auth** — Magic link or password reset flow
- **Plugin auto-update** — Plugin checks for updates from Wursor - **Plugin auto-update** — Plugin checks for updates from Wursor
- **Telemetry** — Minimal events (sign-up, connect, task start, task approve, task reject, deploy) with consent dialog - **Telemetry** — Minimal events (sign-up, connect, task start, task approve, task reject, deploy) with consent dialog
- **Watched first deploys (R14)** — First N deploys of a new account use the stricter health contract
- **Documentation** — `README.md` with install instructions and quickstart - **Documentation** — `README.md` with install instructions and quickstart
- **Bug bash** — Internal team runs through the full flow - **Bug bash** — Internal team runs through the full flow on a sacrificial WordPress site before any strangers
#### Deliverables #### Deliverables
- Web app deployed to staging - Web app deployed to staging
- Plugin packaged for WordPress plugin repo - Plugin packaged for WordPress plugin repo
- Error states all wired - Error states all wired
- Intent chips, starters, and structured reject live
- Minimal telemetry with consent - Minimal telemetry with consent
- `README.md` updated for alpha users - `README.md` updated for alpha users
--- ---
## 5. Phase 2 — Intelligence (Weeks 916) ## 6. Phase 2 — Intelligence (Weeks 916)
| Sprint | Focus | Files | | Sprint | Focus | Files |
|--------|-------|-------| |--------|-------|-------|
| 9 | Plugin playbook (install, configure, fix conflicts) | `api/src/playbooks/plugin.ts` | | 9 | Plugin playbook (catalog only, reputation gate, egress watch, configure, fix conflicts) | `api/src/playbooks/plugin.ts`, `api/src/playbooks/catalog.ts` |
| 10 | Site build playbook (from scratch, limited) | `api/src/playbooks/site-build.ts` | | 10 | Site build playbook (from scratch, limited) | `api/src/playbooks/site-build.ts` |
| 11 | Mobile-responsive preview | `web/src/components/Preview.tsx` | | 11 | Mobile-responsive preview | `web/src/components/Preview.tsx` |
| 12 | Agent clarifying questions | `api/src/services/agent-orchestrator.ts` | | 12 | Agent clarifying questions | `api/src/services/agent-orchestrator.ts` |
| 13 | Visual design picker (theme gallery) | `api/src/playbooks/design.ts`, `web/src/components/DesignPicker.tsx` | | 13 | Visual design picker (theme gallery); pull fork-and-pick forward if Sprint 8 reject rate is high | `api/src/playbooks/design.ts`, `web/src/components/DesignPicker.tsx` |
| 14 | Multi-step workflows (queue changes) | `api/src/services/playbook-runner.ts` | | 14 | Multi-step workflows (queue changes) | `api/src/services/playbook-runner.ts` |
| 15 | Closed alpha with 1020 users | Telemetry review, baselines | | 15 | Closed alpha with 1020 users — must include ≥1 Elementor site and ≥1 managed host (R13). Standby replica spike if mirror p95 missed the 5-min exit. | Telemetry review, baselines |
| 16 | Alpha feedback → Phase 2 exit review | All §11 baselines collected | | 16 | Alpha feedback → Phase 2 exit review | All §11 baselines collected |
Sprint 9 acceptance (R2): agent can install only from the ~40-slug catalog; reputation gate fails closed; unexpected sandbox egress aborts the install; deploy re-checks reputation even after approve. `wp plugin install <url>` remains absent from the tool schema.
--- ---
## 6. TDD Rules ## 7. TDD Rules
1. **Write the test first.** No implementation code is written without a failing test. 1. **Write the test first.** No implementation code is written without a failing test.
2. **One assertion per test.** Each test verifies exactly one behavior. 2. **One assertion per test.** Each test verifies exactly one behavior.
@@ -1028,7 +1200,7 @@ test('new user connects site, makes a content change, previews, approves in ≤5
--- ---
## 7. CI/CD Pipeline ## 8. CI/CD Pipeline
```yaml ```yaml
# .github/workflows/ci.yml — runs on every PR # .github/workflows/ci.yml — runs on every PR
@@ -1090,7 +1262,7 @@ jobs:
--- ---
## 8. Glossary ## 9. Glossary
| Term | Definition | | Term | Definition |
|------|------------| |------|------------|
@@ -1099,7 +1271,11 @@ jobs:
| **Plugin connector** | The WordPress plugin that connects the user's site to Wursor | | **Plugin connector** | The WordPress plugin that connects the user's site to Wursor |
| **Mirror** | The process of copying a site's theme, plugins, content, and settings into a sandbox | | **Mirror** | The process of copying a site's theme, plugins, content, and settings into a sandbox |
| **Deploy** | The process of applying sandbox changes to the live site | | **Deploy** | The process of applying sandbox changes to the live site |
| **Warm pool** | Pre-booted WordPress containers ready to accept a mirror | | **Warm pool** | Paused WordPress images plus a small number of hot spares, ready to accept a task-scoped mirror |
| **Media proxy** | Sandbox nginx rewrite of `/wp-content/uploads/*` to the live origin |
| **Capability tier** | content-safe / design-safe / install-safe, computed at connect |
| **Changeset journal** | Numbered deploy operations; rollback walks them backwards |
| **No-surprise gate** | Blocks slug / visibility / payment / role deploys without an explicit confirm |
| **SSE** | Server-Sent Events — the protocol used to stream agent responses to the frontend | | **SSE** | Server-Sent Events — the protocol used to stream agent responses to the frontend |
--- ---
+171 -21
View File
@@ -185,14 +185,14 @@ Every user request maps to a **playbook** — a structured, multi-step agent wor
#### 7.1.4 WordPress plugin connector #### 7.1.4 WordPress plugin connector
- One-click install from wp-admin plugin directory - One-click install from wp-admin plugin directory
- Pairing flow: user copies a 6-character code from Wursor web app, pastes it into the plugin - Pairing flow: user copies an 8+ character code from Wursor web app (5-minute TTL, 5-attempt lockout, bound to account + site URL), pastes it into the plugin
- Plugin exposes: site info (theme, plugins, content), file system (read/write), database (read/write), WP-CLI (full access) - Plugin exposes: site info (theme, plugins, content), file system (read/write), database (read/write), WP-CLI (full access)
- All communication over HTTPS with token-based auth - All communication over HTTPS with token-based auth
- Plugin auto-updates; no user maintenance - Plugin auto-updates; no user maintenance
#### 7.1.5 Cloud sandbox orchestration #### 7.1.5 Cloud sandbox orchestration
- Spin up a sandbox in ≤ 10 seconds (warm pool) - Spin up a sandbox in ≤ 10 seconds (warm pool)
- Mirror the user's site: theme, plugins, content, media (lazy sync for media) - Mirror the user's site: theme, plugins, content (task-scoped). Media is proxied from origin, not copied.
- Full network access (so the agent can install plugins from the WordPress repo) - Full network access (so the agent can install plugins from the WordPress repo)
- 15-minute idle timeout (auto-hibernate, resume on user interaction) - 15-minute idle timeout (auto-hibernate, resume on user interaction)
- 24-hour hard timeout (sandbox destroyed, no exceptions) - 24-hour hard timeout (sandbox destroyed, no exceptions)
@@ -223,9 +223,11 @@ Every user request maps to a **playbook** — a structured, multi-step agent wor
#### 7.1.10 Safety & trust #### 7.1.10 Safety & trust
- Every change is previewed before apply — no "apply now, preview later" - Every change is previewed before apply — no "apply now, preview later"
- Agent has a "no-surprise" rule: it must surface any action that costs money (e.g., a paid plugin) or affects SEO (e.g., URL changes) - Agent has a "no-surprise" rule: it must surface any action that costs money (e.g., a paid plugin) or affects SEO (e.g., URL changes)
- That rule is mechanical at deploy, not just conversational: URL/slug changes, `blog_public`, payment/shipping, and user/role table writes are blocked until the user confirms that specific bullet (R12)
- Agent role: "I changed your homepage layout. It also removed your sidebar widget. Is that OK?" - Agent role: "I changed your homepage layout. It also removed your sidebar widget. Is that OK?"
- Deploy history: a timeline of all changes, with one-click undo per change - Deploy history: a timeline of all changes, with one-click undo per change
- Undo reverts the last deploy (not individual file changes — the user sees "your site has been restored to before that change") - Undo reverts the last deploy (not individual file changes — the user sees "your site has been restored to before that change")
- The last 3 deploy snapshots are stored in Wursor's cloud as well as on the site, so Undo still works if the live site is down (R3)
### 7.2 P1 — Follow-on ### 7.2 P1 — Follow-on
@@ -349,7 +351,7 @@ Every user request maps to a **playbook** — a structured, multi-step agent wor
- **Site mirroring:** - **Site mirroring:**
- Plugin list and active theme → installed immediately - Plugin list and active theme → installed immediately
- Content (posts, pages, options) → pulled from the live site via the plugin API - Content (posts, pages, options) → pulled from the live site via the plugin API
- Media files → lazy sync; only pulled when the preview or agent accesses them - Media files → not copied. Sandbox nginx proxies `/wp-content/uploads/*` to the live origin (or a signed Wursor proxy). A file is pulled only when the agent replaces it.
- **Networking:** Sandboxes have full outbound internet access (for plugin installs, API calls). No inbound access except from the Wursor API server. - **Networking:** Sandboxes have full outbound internet access (for plugin installs, API calls). No inbound access except from the Wursor API server.
- **Idle timeout:** 15 minutes. User typing or viewing the preview resets the timer. - **Idle timeout:** 15 minutes. User typing or viewing the preview resets the timer.
- **Hard timeout:** 24 hours. Sandbox is destroyed regardless of state. - **Hard timeout:** 24 hours. Sandbox is destroyed regardless of state.
@@ -364,9 +366,11 @@ When the user approves:
3. **Database changes** — send SQL migration to the plugin's deploy API (or WP-CLI commands) 3. **Database changes** — send SQL migration to the plugin's deploy API (or WP-CLI commands)
4. **Plugin changes** — plugin installs/activations sent as WP-CLI commands 4. **Plugin changes** — plugin installs/activations sent as WP-CLI commands
5. **Verify** — plugin confirms the live site is functional after changes 5. **Verify** — plugin confirms the live site is functional after changes
6. **Snapshot** — deploy snapshot stored for rollback (files + DB state) 6. **Snapshot** — deploy snapshot stored for rollback (files + DB state) on the site **and** in Wursor's cloud (last 3). Cloud copy is what Undo uses if the live site is down.
7. **Drift check** — re-hash the live files/options in the changeset at approve time. If the live site changed since the preview, ask before clobbering (R11).
8. **No-surprise gate** — slug / `blog_public` / payment / role changes require an explicit confirm bullet (R12).
Rollback restores the files and database from the snapshot. Rollback walks the changeset journal backwards, then restores from the snapshot if the journal is incomplete.
### 8.5 Error & offline states ### 8.5 Error & offline states
@@ -432,14 +436,20 @@ Rollback restores the files and database from the snapshot.
### Phase 0 — Pivot & spec (now) ### Phase 0 — Pivot & spec (now)
- Rewrite PRD for non-technical-first - Rewrite PRD for non-technical-first
- Spike: cloud sandbox orchestration (WordPress in Docker, warm pool, site mirroring) - Risk rewrite: §13 is 14 risks with mitigations that map to tests or UI (this revision)
- Spike: golden-task eval harness (20 prompts × ≥2 canned WP sites, scored)
- Spike: Elementor / builder detection in site-info
- Spike: pairing threat model (8+ char, TTL, lockout, HMAC, scoped tokens)
- Spike: time a task-scoped content mirror + media proxy against a ≥2GB site
- Spike: cloud sandbox orchestration (WordPress in Docker, overlay + pause pool)
- Spike: WordPress plugin (REST API, file read/write, DB access, WP-CLI) - Spike: WordPress plugin (REST API, file read/write, DB access, WP-CLI)
- Spike: basic chat + preview web app - Spike: basic chat + preview web app
- Lock the P0 plugin catalog (~40 slugs) before any agent can install
### Phase 1 — Foundation (weeks 18) ### Phase 1 — Foundation (weeks 18)
- Web app: sign-up, site connection (plugin auth), chat, preview, approve/reject - Web app: sign-up, site connection (plugin auth), chat, preview, approve/reject
- Sandbox infrastructure: warm pool, site mirroring, idle timeout, GC - Sandbox infrastructure: overlay + pause-to-disk warm pool, task-scoped mirroring, media proxy, idle timeout, GC
- WordPress plugin: site info API, deploy receiver, rollback, auto-update - WordPress plugin: site info API (builder + capability tiers + pre-flight), HMAC-signed pairing, two-phase deploy receiver, journaled rollback, auto-update
- Playbooks: content edit (text, images, pages), design change (layout, colors) - Playbooks: content edit (text, images, pages), design change (layout, colors)
- Deploy history: timeline, one-click undo - Deploy history: timeline, one-click undo
@@ -473,16 +483,149 @@ Rollback restores the files and database from the snapshot.
## 13. Risks & Mitigations ## 13. Risks & Mitigations
| Risk | Impact | Mitigation | Isolation, a warning dialog, and a fallback model are not mitigations. Each row below maps to a test, a playbook constraint, or a product surface. Residual impact assumes the mitigation ships; **Launch-blocking** means Phase 1 cannot exit without it.
### 13.1 Documented risks
#### R1 — Agent breaks the sandbox site
| | |
| :--- | :--- |
| **Impact (original → residual)** | Medium → Low for the live site; **High for session trust** until checkpoints ship |
| **Launch-blocking** | Circuit breaker and per-step verify. Full self-heal can follow. |
| **Why the old mitigation fails** | “GC and start fresh” keeps the live site safe and kills the session. A white-screen preview after 90 seconds is a churn event. Retrying the same trajectory also burns the warm pool and the model budget. |
| **Mitigation** | Copy-on-write checkpoint after every successful playbook step (overlayfs / Docker commit). Verify after every tool call (target URL 200, no new PHP fatal, `siteurl`/`home` unchanged). Two consecutive verify failures → circuit breaker, stop, and talk — do not thrash. Content playbooks prefer WP-CLI / REST (`wp post update`) over theme file surgery. A tiny internal self-heal path rewinds the last checkpoint before the user sees a dead preview. |
| **Owner / where** | Sprint 1 (overlay + GC), Sprint 3 (circuit breaker), Sprint 4 (REST-first content), Sprint 6 (reuse verifier on the sandbox). |
#### R2 — Agent installs a malicious plugin
| | |
| :--- | :--- |
| **Impact (original → residual)** | Medium → Low **if** the catalog and gates ship. Deploy path makes the unmitigated case High. |
| **Launch-blocking** | Tool-schema allowlist in Sprint 3. Catalog + reputation in Sprint 9, but the agent must not be able to `wp plugin install <url>` before then. |
| **Why the old mitigation fails** | wordpress.org is not a reviewed-safe catalog. Sandbox isolation only holds until the user clicks “Looks good → Apply.” A pretty form that phones home looks fine in preview. |
| **Mitigation** | P0/P1 install from a curated catalog of ~40 slugs only (CF7, WPForms, Yoast, RankMath, Woo, Elementor, WooPayments, …). Reputation gate before install: WPScan / Patchstack CVE, last updated < 18 months, `tested up to` within 2 majors, active-install floor, zip SHA against a Wursor mirror. Fail closed. While a new plugin first activates, sandbox egress is allowlisted (wordpress.org, gravatar, the users own domain); unexpected egress is surfaced in chat and the plugin is not applied. Static smell test rejects `eval(base64_decode`, `/e` preg, unexpected remote `file_get_contents`. Deploy re-checks reputation even after approve. No arbitrary ZIP, no premium marketplace URLs, no `wp plugin install <url>` in v1. |
| **Owner / where** | Sprint 3 (`tool-schemas.ts` allowlist), Sprint 9 (catalog + reputation + egress watch). |
#### R3 — Deploy to live site fails
| | |
| :--- | :--- |
| **Impact (original → residual)** | High → Medium (hosts stay messy) |
| **Launch-blocking** | **Yes** |
| **Why the old mitigation fails** | The failure mode is a **partial** deploy, not a clean 500. CSS lands, `wp_options` is half-updated, object cache serves the old theme, then rollback fails because the customers disk is full. HTTP 200 does not catch a wrong `home` URL or a dropped menu. |
| **Mitigation** | Two-phase deploy: prepare artifacts under `wp-content/upgrade/wursor-<id>/`, then commit. Prepare failure = live site untouched. Pre-flight before the confirm dialog: disk free, `ABSPATH` writable, `DISALLOW_FILE_MODS` off, PHP memory, `post_max_size`, can flush object/opcode cache, can enter maintenance mode. Failure copy is host-ticket language, not “deploy failed.” Every file / option / WP-CLI call is a numbered journal entry; rollback walks it backwards. Health contract: homepage + one inner page + `/wp-json` + `wp-login.php` + no new fatal + `siteurl`/`home` match intent + screenshot SSIM vs sandbox. Last 3 snapshots live in **Wursors cloud**, not only on the customer disk, so Undo still works when the site is down. Maintenance mode for the commit window (seconds). Detect managed hosts at connect; v1 still file-pushes but must purge known caches. Canary rewrite (`/?wursor_canary=`) is stretch; journal + cloud snapshot is the P0 version. |
| **Owner / where** | Sprint 2 (pre-flight on site-info), Sprint 6 (two-phase, journal, cloud snapshot, health contract). The existing `handles partial failures` test is acceptance, not a comment. |
#### R4 — Mirroring a large site is slow
| | |
| :--- | :--- |
| **Impact (original → residual)** | High → Medium once thin-slice + media proxy ship (standby replica is Phase 2) |
| **Launch-blocking** | **Yes** for task-scoped mirror + media proxy. Standby replica is not. |
| **Why the old mitigation fails** | Warm pool absorbs *boot*, not *copy*. “Incremental” was unnamed. The 5-minute Phase 1 exit criterion dies on a Woo store with 8 years of posts. Media is often not the bottleneck — `wp_posts`, plugin folders, and upload thumbs are. |
| **Mitigation** | **Task-scoped mirrors.** Content edit → target posts + `wp_options` + menus + active theme/plugins (skip orders, logs, transients, revisions). Design → theme + `theme.json` + templates + a few representative pages. Plugin install → current site + the new plugin, still skip Woo order tables. Theme swap / rebuild → fuller clone. **Remote media proxy:** sandbox nginx rewrites `/wp-content/uploads/*` to the live origin (or a signed Wursor proxy). Copy a file only when the agent replaces it. Plugin sends a path→sha256 manifest; wordpress.org packages come from a Wursor cache, not the customer host. Chunked resumable zstd batches over plugin REST, not one zip. Default DB exclude: `_transient_*`, action scheduler logs, Woo orders/customers for non-commerce playbooks. Progressive preview: show the target page as soon as *that* page is mirrored. **Phase 2:** a per-site standby replica synced by webhook / 15-min cron so a task is `fork replica`, not `pull production`. |
| **Owner / where** | Sprint 1 (proxy, hash manifest, subset dump), §14 media decision, Phase 2 standby replica. |
#### R5 — User can't describe what they want
| | |
| :--- | :--- |
| **Impact (original → residual)** | Medium → Medium. Unmitigated this is **High for activation** — vague language is the default for the primary persona. |
| **Launch-blocking** | Intent chips, site-aware starters, and structured reject in Sprint 8. Full design-fork picker can wait on alpha reject rate. |
| **Why the old mitigation fails** | Clarifying questions feel like a support form. The §11 “claps back” metric (reject → re-describe ≤ 20%) *is* this risk. |
| **Mitigation** | Empty state is intent chips (“Change wording” / “New look” / “Add a form” / “Somethings broken”), not a blank chat. After connect, scan for default theme H1, missing favicon, no contact page, mobile overflow, and offer three *specific* starter sentences. Reject is chips (“Wrong color” / “Too busy” / “Keep my logo” / “Undo only the last thing”), each mapped to a constrained follow-up. Vague design prompts should *show* 23 cheap visual forks (theme.json / CSS / catalog themes) rather than ask “modern or classic?” Point-and-talk: click an element in the preview, then type “make this blue” — Phase 1 spike, not Phase 4 magic. Before/after slider on the preview. Voice input on mobile via browser SpeechRecognition is enough for alpha. |
| **Owner / where** | Sprint 8 (chips, starters, structured reject), Phase 1 spike (point-and-talk), Sprint 12 (disambiguation), pull a fork-and-pick slice forward from Sprint 13 if design-prompt reject rate is high. |
#### R6 — Plugin compatibility (old WordPress, old PHP)
| | |
| :--- | :--- |
| **Impact (original → residual)** | Medium → Medium |
| **Launch-blocking** | Version matrix + capability tiers at connect (Sprint 2). |
| **Why the old mitigation fails** | A warning a dentist cannot act on is a bounce. The real incompatibilities are Elementor vs Gutenberg vs Classic, security plugins that kill REST, and hosts that set `DISALLOW_FILE_MODS` — not PHP 7.4 vs 8.2 alone. |
| **Mitigation** | **Capability tiers, not a binary supported flag.** *Content-safe:* edit posts/pages via REST (always try to offer this). *Design-safe:* writable theme / `theme.json` / page-builder API. *Install-safe:* WP-CLI or filesystem and `DISALLOW_FILE_MODS` off. Chat: “I can change text on your site today. Installing plugins needs a WordPress update — I can preview that first if you want.” **P0 matrix (locked):** WP 6.1+ and PHP 8.0+ for full playbooks; WP 5.86.0 / PHP 7.4 are content-only. Publish this in the plugin readme. Detect Elementor / Beaver / Divi / Gutenberg at connect; content and layout playbooks must use the matching adapter or they will edit `post_content` while Elementor JSON is what renders. Sandbox may run newer PHP than production; deploy lints PHP against the live interpreter version. “Update WordPress for me” is a sandbox-then-approve playbook, not a warning. If PHP is old, generate a 4-line host-ticket email the user can forward. Connect-time scan also flags Wordfence / iThemes / disabled REST, `DISALLOW_FILE_EDIT`, `open_basedir`, object cache. |
| **Owner / where** | Phase 0 spike (matrix + builder detect), Sprint 2 (tiered connect + concierge copy), Sprint 45 (adapters for builders we actually see). |
#### R7 — Grok model quality for agentic tasks
| | |
| :--- | :--- |
| **Impact (original → residual)** | Medium → Medium. Unmitigated this is High until the eval harness has a score. |
| **Launch-blocking** | Golden-task harness (Phase 0) + tool allowlist (Sprint 3). |
| **Why the old mitigation fails** | “Have a fallback model” is not a design. WordPress is full of hallucinated WP-CLI flags. Grok stays the default *vendor*; it must not be the architecture. |
| **Mitigation** | Playbooks are the intelligence; the model fills slots (`page`, `old`, `new`) and a deterministic runner executes. Almost never emit free-form shell. Allowlisted tools only: a short WP-CLI list, REST routes, file paths under `wp-content/themes/{active}` and `wp-content/uploads`. Forbidden: `wp db query` with DROP/TRUNCATE, `wp eval`, `wp config`, `rm`, writing `wp-config.php`, touching `mu-plugins`. Golden-task harness from day 1: 2050 fixture sites × ~20 prompts, asserting preview text / options / screenshot — not “the model said it worked.” Router: cheap/fast model classifies + asks one question; stronger model (Grok or fallback) only for design/plugin reasoning. Fallback is **per playbook**. Sprint 3 client is `llm-client.ts` with a Grok adapter; vendor switch is an env var. Self-critique before the user sees the preview (health contract + screenshot: “Did the H1 actually change?”). Store anonymized winning tool traces as few-shots. If the classifier is unsure, offer chips (R5) — do not improvise a theme rewrite. |
| **Owner / where** | Phase 0 (eval harness, before playbooks), Sprint 3 (allowlist + provider-agnostic client), Sprint 45 (slot-filling). |
#### R8 — Sandbox cost scales with usage
| | |
| :--- | :--- |
| **Impact (original → residual)** | Low compute → Low. **Token spend is the real risk** (Medium) and was previously named but not mitigated. |
| **Launch-blocking** | Per-task token budget in Sprint 3. Pause-to-disk in Sprint 1. |
| **Why the old mitigation fails** | The old row admitted API spend is higher than VPS hours, then did nothing about it. A warm pool of *running* WP+MySQL+Redis boxes is idle burn. One runaway tool loop can cost more than a month of that users subscription. |
| **Mitigation (compute)** | 15-min idle → pause / checkpoint to disk, resume in ~2s. Warm pool = paused images + 12 hot spares per region, not 510 fully running. Shared read-only WordPress image + overlayfs site layers. Predictive pool on time-of-day / queue depth. Do not copy media (R4). Content-edit without a full container is stretch. |
| **Mitigation (tokens — the actual cost)** | Hard cap per task: e.g. 12 tool rounds and a dollar token budget. Hit the cap → R1 circuit breaker, not another retry. Playbook-specific short prompts; do not dump the entire site-info blob every turn. Cache prompt prefixes and repeated tool results (“whats the active theme”). Price the unit against p95 token usage, not sandbox hours. Free tier = 1 concurrent sandbox + M token-budgeted tasks (kills sandbox-farming). |
| **Owner / where** | Sprint 1 (pause + overlay + no media copy), Sprint 3 (token budget), Phase 3 pricing (unit = task budget). |
### 13.2 Risks the original table did not name
The spec creates these. They are in scope for Phase 1.
#### R9 — Plugin is a privileged backdoor
Plugin exposes files + DB + WP-CLI. A 6-character pairing code (`[A-Z0-9]{6}`) with no stated rate limit, plus a stolen token, is site ownership.
**Mitigation:** Pairing codes are 8+ characters, 5-minute TTL, 5-attempt lockout, bound to account + site URL. Tokens are hashed at rest, scoped (read vs deploy), rotatable, stored in WP encrypted with a site salt. Requests are HMAC-signed (body + timestamp) so a leaked URL is not enough. **Launch-blocking. Sprint 2.**
#### R10 — Sandbox holds real customer PII
A full mirror copies Woo orders, emails, form submissions, and license keys into a container with outbound internet.
**Mitigation:** Default DB subset excludes orders, customers, and form entries. Redact options matching `*_key`, `*_secret`, `smtp_pass`. Egress allowlist (R2). 24-hour hard delete already helps; pin sandbox region to the users site region when we have it, and document retention. **Launch-blocking for subset + redact. Sprint 1.**
#### R11 — Live-site drift during a session
An agency or the user edits wp-admin while the sandbox is open. Deploy silently overwrites their work.
**Mitigation:** On approve, re-hash the live files/options in the changeset. If they drifted, show “your live site changed since this preview — refresh preview or apply anyway.” Never silent clobber. **Sprint 6.**
#### R12 — Visual-only approve misses silent damage
The user never sees a diff (product principle — keep it). That is right for UX and wrong for SEO slugs, `robots`, deleted pages, and payment settings.
**Mitigation:** Keep the UI visual. The agent already owes a plain-language change list (“I also removed your sidebar”). Make the no-surprise rule mechanical: block deploy on URL/slug changes, `blog_public`, payment/shipping, and user/role tables unless the user explicitly confirms that bullet. **Launch-blocking. Sprint 6.**
#### R13 — Page builders, managed hosts, and security plugins
Will cause more alpha failures than Grok quality. Partly covered by R6.
**Mitigation:** Connect-time detection + adapters + capability tiers (R6). Alpha recruitment must include at least one Elementor site and one managed host (WP Engine / Kinsta / SiteGround), not only Twenty Twenty-Four on a VPS. **Alpha plan, Sprint 2 detect, Sprint 45 adapters.**
#### R14 — One bad live deploy kills trust
A restaurant homepage going down on a Saturday is a tweet. Trust is the moat.
**Mitigation:** R3 journal + cloud snapshots + auto-rollback. First N deploys of a new account, and any theme/plugin install, take a slower “Wursor is watching this deploy” path with the stricter health contract. Dogfood on a sacrificial WordPress site before any strangers. **Launch-blocking process. Sprint 6 + Sprint 8 bug bash.**
### 13.3 Residual ratings (if the mitigations ship)
| ID | Residual impact | Launch-blocking? |
| :--- | :--- | :--- | | :--- | :--- | :--- |
| Agent breaks the sandbox site | Medium | Sandbox is ephemeral; worst case, GC and start fresh. Live site never touched. | | R1 Sandbox self-heal | Low | Circuit breaker yes; full self-heal no |
| Agent installs a malicious plugin | Medium | Plugin repo is reviewed; sandbox is isolated; no data leaks to the live site. | | R2 Malicious plugin | Low if catalog + gate ship | **Yes** — tool allowlist |
| Deploy to live site fails | High | Plugin detects failure, rolls back automatically, sandbox stays alive for retry. | | R3 Deploy / rollback | Medium (hosts are messy) | **Yes** |
| Mirroring a large site is slow | High | Lazy sync for media; incremental content sync; warm pool absorbs the variance. | | R4 Slow mirror | Medium (standby is Phase 2) | **Yes** — thin-slice + media proxy |
| User can't describe what they want | Medium | Agent asks clarifying questions; suggests options ("Would you like a modern look or a classic look?"). | | R5 Intent UX | Medium | Chips + starters in Sprint 8 |
| Plugin compatibility (old WordPress, old PHP) | Medium | Detect at connection time; warn the user; support the top 90% of versions. | | R6 Compatibility | Medium | Matrix + tiers in Sprint 2 |
| Grok model quality for agentic tasks | Medium | Evaluate in Phase 0 spike; have a fallback model path (switch to Claude or GPT-4o). | | R7 Model quality | Medium | Harness + allowlist in Phase 0 / Sprint 3 |
| Sandbox cost scales with usage | Low | ~$0.02/task at v1 volume; even at 100k tasks/month, < $5k. Agent API calls are the higher cost. | | R8 Cost | Low compute / Medium tokens | Token budget in Sprint 3 |
| R9 Plugin auth | High if skipped | **Yes** |
| R10 PII in sandbox | High if skipped | **Yes** — subset + redact |
| R11 Drift | Medium | Sprint 6 |
| R12 Silent damage | High if skipped | **Yes** — no-surprise gate |
| R13 Builders / hosts | Medium | Alpha plan, not just code |
| R14 Trust-ending deploy | High | Dogfood + watched first deploys |
--- ---
@@ -500,10 +643,13 @@ Rollback restores the files and database from the snapshot.
### Remaining (genuinely open) ### Remaining (genuinely open)
1. **Free tier limits:** How many tasks per month before asking for payment? Set during Phase 3 beta. 1. **Free tier limits:** How many token-budgeted tasks per month before asking for payment? Unit is a task budget (R8), not sandbox hours. The number itself is set during Phase 3 beta.
2. **Pricing:** Final $X and free tier limits set during Phase 3 paid beta. 2. **Pricing:** Final $X and free tier limits set during Phase 3 paid beta.
3. **Media library handling:** Lazy sync is the plan, but large media libraries (20GB+) need specific design. Phase 1 spike.
4. **Plugin compatibility:** Which WordPress + PHP versions are we guaranteeing? Phase 0 spike. ### Locked this revision (were open in v2.0)
3. **Media library handling:** Remote media proxy is the default. Sandbox nginx rewrites `/wp-content/uploads/*` to the live origin (or a signed Wursor proxy). A file is copied into the sandbox only when the agent replaces it. Full-library copy is not a v1 path, including for 20GB+ libraries. Task-scoped DB/file mirrors (R4) plus this proxy are how the 5-minute exit criterion stays honest.
4. **Plugin compatibility:** P0 guarantee is WP 6.1+ and PHP 8.0+ for full playbooks. WP 5.86.0 and PHP 7.4 are **content-only** (REST edits, no theme file writes, no plugin installs). Older than that is unsupported with concierge copy to the host. Capability is tiered at connect (content-safe / design-safe / install-safe), not a binary block (R6). Page-builder detection (Elementor, Beaver, Divi, Gutenberg) is part of connect, not a later surprise.
--- ---
@@ -515,7 +661,11 @@ Rollback restores the files and database from the snapshot.
- **Plugin connector** — The WordPress plugin that connects the user's site to Wursor - **Plugin connector** — The WordPress plugin that connects the user's site to Wursor
- **Mirror** — The process of copying a site's theme, plugins, content, and settings into a sandbox - **Mirror** — The process of copying a site's theme, plugins, content, and settings into a sandbox
- **Deploy** — The process of applying sandbox changes to the live site - **Deploy** — The process of applying sandbox changes to the live site
- **Warm pool** — Pre-booted WordPress containers ready to accept a mirror, reducing spin-up time - **Warm pool** — Paused WordPress images plus a small number of hot spares, ready to accept a task-scoped mirror
- **Media proxy** — Sandbox nginx rewrite of `/wp-content/uploads/*` to the live origin so previews work without copying the library
- **Capability tier** — What Wursor can do on this site today: content-safe, design-safe, and/or install-safe
- **Changeset journal** — Numbered list of file, option, and WP-CLI operations that deploy walks forward and rollback walks backward
- **No-surprise gate** — Deploy blocker for slug, visibility, payment, and role changes until the user confirms that bullet
### B. P0 playbook sketches ### B. P0 playbook sketches
1. **Edit text** — parse user request → find content in DB → update → verify page loads → show preview 1. **Edit text** — parse user request → find content in DB → update → verify page loads → show preview
+2 -1
View File
@@ -10,6 +10,7 @@ Just describe what you want. Wursor does the rest.
- Full product spec: [PRD.md](./PRD.md) - Full product spec: [PRD.md](./PRD.md)
- Build guide (TDD, sprints, CI): [IMPLEMENTATION.md](./IMPLEMENTATION.md) - Build guide (TDD, sprints, CI): [IMPLEMENTATION.md](./IMPLEMENTATION.md)
- Phase 0 gate: [spikes/README.md](./spikes/README.md)
## What it does ## What it does
@@ -22,4 +23,4 @@ No code. No terminals. No wp-admin.
## Status ## Status
Spec / Phase 0. Implementation has not started. Spec / Phase 0. Workspace folders exist. Product code waits on [spikes/](./spikes/README.md).
View File
+6
View File
@@ -0,0 +1,6 @@
{
"name": "@wursor/api",
"private": true,
"type": "module",
"scripts": {}
}
View File
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src",
"types": ["node"]
},
"include": ["src"]
}
View File
+20
View File
@@ -0,0 +1,20 @@
# Golden-task harness
Phase 0. Two canned sites, 20 prompts, builder detect, subset + media proxy.
```
e2e/golden/
prompts.json
sites/gutenberg-business/site.json
sites/elementor-restaurant/site.json
src/ # detect, apply, score, subset, proxy, runners
__tests__/
runs/latest.json
runs/mirror-timing.json
```
```bash
pnpm test:e2e
pnpm --filter @wursor/e2e golden # 20/20 fixture traces; live Grok if XAI_API_KEY
pnpm --filter @wursor/e2e mirror:time # ≥2GB synthetic + p50/p95
```
@@ -0,0 +1,72 @@
import { describe, it, expect } from 'vitest';
import { detectBuilder } from '../src/builder-detect.ts';
describe('detectBuilder', () => {
it('reports elementor when the plugin is active and post meta has _elementor_edit_mode', () => {
expect(
detectBuilder({
theme: 'hello-elementor',
plugins: [{ slug: 'elementor', active: true }],
posts: [{ id: 1, slug: 'home', title: 'Home', content: '', meta: { _elementor_edit_mode: 'builder' } }],
}),
).toBe('elementor');
});
it('reports beaver when beaver-builder is active and _fl_builder_data is present', () => {
expect(
detectBuilder({
theme: 'bb-theme',
plugins: [{ slug: 'beaver-builder-lite-version', active: true }],
posts: [{ id: 1, slug: 'home', title: 'Home', content: '', meta: { _fl_builder_data: '{}' } }],
}),
).toBe('beaver');
});
it('reports divi when the theme is Divi and _et_pb_use_builder is on', () => {
expect(
detectBuilder({
theme: 'Divi',
plugins: [],
posts: [{ id: 1, slug: 'home', title: 'Home', content: '', meta: { _et_pb_use_builder: 'on' } }],
}),
).toBe('divi');
});
it('reports gutenberg when content has block markup and no builder plugin', () => {
expect(
detectBuilder({
theme: 'twentytwentyfour',
plugins: [],
posts: [{ id: 1, slug: 'home', title: 'Home', content: '<!-- wp:heading --><h1>Hi</h1><!-- /wp:heading -->', meta: {} }],
}),
).toBe('gutenberg');
});
it('reports classic when content is HTML and no builder plugin is active', () => {
expect(
detectBuilder({
theme: 'twentytwentyone',
plugins: [],
posts: [{ id: 1, slug: 'home', title: 'Home', content: '<h1>Hi</h1>', meta: {} }],
}),
).toBe('classic');
});
it('prefers elementor over gutenberg markup when both are present', () => {
expect(
detectBuilder({
theme: 'hello-elementor',
plugins: [{ slug: 'elementor', active: true }],
posts: [
{
id: 1,
slug: 'home',
title: 'Home',
content: '<!-- wp:heading --><h1>Hi</h1><!-- /wp:heading -->',
meta: { _elementor_data: '[]' },
},
],
}),
).toBe('elementor');
});
});
+54
View File
@@ -0,0 +1,54 @@
import { describe, it, expect } from 'vitest';
import { exportDbSubset } from '../src/subset.ts';
import { mediaProxyTarget, stageReplacement } from '../src/media-proxy.ts';
import type { SiteExport } from '../src/types.ts';
const dump = (): SiteExport => ({
origin: 'https://example.com',
tables: {
wp_posts: [{ ID: 1, post_title: 'Home' }],
wp_postmeta: [{ post_id: 1, meta_key: '_edit_lock', meta_value: '1' }],
wp_options: [
{ option_name: 'blogname', option_value: 'Biz' },
{ option_name: 'woocommerce_stripe_secret_key', option_value: 'sk_live_xxx' },
{ option_name: 'smtp_pass', option_value: 'secret' },
],
wp_wc_orders: [{ id: 99, total: '40.00' }],
wp_comments: [{ comment_ID: 1, comment_content: 'hi' }],
},
uploads: [{ path: '/wp-content/uploads/2024/hero.jpg', bytes: 2_147_483_648 }],
});
describe('exportDbSubset', () => {
it('keeps posts, postmeta, and options for a content playbook', () => {
expect(exportDbSubset(dump(), { playbook: 'content', postIds: [1] }).tables).toEqual(
expect.arrayContaining(['wp_posts', 'wp_postmeta', 'wp_options']),
);
});
it('drops Woo orders from a content-edit slice', () => {
expect(exportDbSubset(dump(), { playbook: 'content', postIds: [1] }).tables).not.toContain('wp_wc_orders');
});
it('drops comments from a content-edit slice', () => {
expect(exportDbSubset(dump(), { playbook: 'content', postIds: [1] }).tables).not.toContain('wp_comments');
});
it('redacts option names ending in _key, _secret, or smtp_pass', () => {
const options = exportDbSubset(dump(), { playbook: 'content', postIds: [1] }).options;
expect(options.every((name) => !/(_key|_secret|smtp_pass)$/.test(name))).toBe(true);
});
});
describe('media proxy', () => {
it('does not copy the uploads library', () => {
expect(mediaProxyTarget(dump(), '/wp-content/uploads/2024/hero.jpg')).toBe(
'https://example.com/wp-content/uploads/2024/hero.jpg',
);
});
it('copies a file only when it is replaced', () => {
const staged = stageReplacement(dump(), '/wp-content/uploads/2024/hero.jpg', 12);
expect(staged.copiedPaths).toEqual(['/wp-content/uploads/2024/hero.jpg']);
});
});
+22
View File
@@ -0,0 +1,22 @@
import { describe, it, expect } from 'vitest';
import { loadPrompts } from '../src/load-prompts.ts';
import { loadSite } from '../src/load-site.ts';
describe('golden prompts', () => {
it('loads twenty prompts', () => {
expect(loadPrompts()).toHaveLength(20);
});
it('covers at least two site fixtures', () => {
expect(new Set(loadPrompts().map((p) => p.site)).size).toBeGreaterThanOrEqual(2);
});
it('gives every prompt a preview_text, option, or screenshot assertion', () => {
const types = new Set(loadPrompts().map((p) => p.assert.type));
expect([...types].every((t) => t === 'preview_text' || t === 'option' || t === 'screenshot')).toBe(true);
});
it('points every prompt at a site fixture that exists', () => {
expect(loadPrompts().every((p) => loadSite(p.site).id === p.site)).toBe(true);
});
});
+86
View File
@@ -0,0 +1,86 @@
import { describe, it, expect } from 'vitest';
import { applyTool } from '../src/apply-tool.ts';
import { checkAssertion } from '../src/check-assertion.ts';
import { scoreGrokResponse } from '../src/score.ts';
import type { SiteFixture } from '../src/types.ts';
const site = (): SiteFixture => ({
id: 'gutenberg-business',
theme: 'twentytwentyfour',
plugins: [],
wordpressVersion: '6.6',
phpVersion: '8.2',
options: { blogname: 'Old Biz', blog_public: '1' },
posts: [
{
id: 1,
slug: 'homepage',
title: 'Home',
content: '<!-- wp:heading --><h1>Welcome to our site</h1><!-- /wp:heading -->',
meta: {},
},
],
uploads: [],
});
describe('applyTool', () => {
it('replaces the homepage heading', () => {
const next = applyTool(site(), {
name: 'edit_heading',
arguments: { page: 'homepage', newText: 'Welcome to My Business' },
});
expect(next.posts[0]?.content).toContain('Welcome to My Business');
});
});
describe('checkAssertion', () => {
it('passes preview_text when the page contains the string', () => {
const next = applyTool(site(), {
name: 'edit_heading',
arguments: { page: 'homepage', newText: 'Welcome to My Business' },
});
expect(
checkAssertion(next, { type: 'preview_text', page: 'homepage', contains: 'Welcome to My Business' }).ok,
).toBe(true);
});
it('fails preview_text when the string is absent', () => {
expect(
checkAssertion(site(), { type: 'preview_text', page: 'homepage', contains: 'Welcome to My Business' }).ok,
).toBe(false);
});
it('passes option when the value matches', () => {
const next = applyTool(site(), {
name: 'update_option',
arguments: { key: 'blogname', value: 'My Business' },
});
expect(checkAssertion(next, { type: 'option', key: 'blogname', value: 'My Business' }).ok).toBe(true);
});
});
describe('scoreGrokResponse', () => {
it('passes a Grok tool-call that satisfies the assertion', () => {
const verdict = scoreGrokResponse({
site: site(),
assert: { type: 'preview_text', page: 'homepage', contains: 'Welcome to My Business' },
grok: {
choices: [
{
message: {
tool_calls: [
{
function: {
name: 'edit_heading',
arguments: JSON.stringify({ page: 'homepage', newText: 'Welcome to My Business' }),
},
},
],
},
},
],
},
});
expect(verdict.passed).toBe(true);
});
});
+122
View File
@@ -0,0 +1,122 @@
[
{
"id": "gb-01",
"site": "gutenberg-business",
"prompt": "Change the homepage heading to Welcome to My Business",
"assert": { "type": "preview_text", "page": "homepage", "contains": "Welcome to My Business" }
},
{
"id": "gb-02",
"site": "gutenberg-business",
"prompt": "Change the site title to Harbor Dental",
"assert": { "type": "option", "key": "blogname", "value": "Harbor Dental" }
},
{
"id": "gb-03",
"site": "gutenberg-business",
"prompt": "Change the About heading to About the practice",
"assert": { "type": "preview_text", "page": "about", "contains": "About the practice" }
},
{
"id": "gb-04",
"site": "gutenberg-business",
"prompt": "On the homepage, replace Family dentistry with Gentle dentistry",
"assert": { "type": "preview_text", "page": "homepage", "contains": "Gentle dentistry" }
},
{
"id": "gb-05",
"site": "gutenberg-business",
"prompt": "Hide the site from search engines",
"assert": { "type": "option", "key": "blog_public", "value": "0" }
},
{
"id": "gb-06",
"site": "gutenberg-business",
"prompt": "Change the contact page heading to Call us today",
"assert": { "type": "preview_text", "page": "contact", "contains": "Call us today" }
},
{
"id": "gb-07",
"site": "gutenberg-business",
"prompt": "Set the tagline to Dentistry for the whole family",
"assert": { "type": "option", "key": "blogdescription", "value": "Dentistry for the whole family" }
},
{
"id": "gb-08",
"site": "gutenberg-business",
"prompt": "Change the services heading to What we offer",
"assert": { "type": "preview_text", "page": "services", "contains": "What we offer" }
},
{
"id": "gb-09",
"site": "gutenberg-business",
"prompt": "Make the homepage heading Harbor Dental Home. I will check the preview.",
"assert": { "type": "screenshot", "page": "homepage", "contains": "Harbor Dental Home" }
},
{
"id": "gb-10",
"site": "gutenberg-business",
"prompt": "Change the About heading to Meet the dentist",
"assert": { "type": "preview_text", "page": "about", "contains": "Meet the dentist" }
},
{
"id": "el-01",
"site": "elementor-restaurant",
"prompt": "Change the homepage heading to Evening table available",
"assert": { "type": "preview_text", "page": "homepage", "contains": "Evening table available" }
},
{
"id": "el-02",
"site": "elementor-restaurant",
"prompt": "Change the site title to Nonna's Kitchen",
"assert": { "type": "option", "key": "blogname", "value": "Nonna's Kitchen" }
},
{
"id": "el-03",
"site": "elementor-restaurant",
"prompt": "Change the menu heading to This week's plates",
"assert": { "type": "preview_text", "page": "menu", "contains": "This week's plates" }
},
{
"id": "el-04",
"site": "elementor-restaurant",
"prompt": "On the homepage, replace Book now with Reserve a table",
"assert": { "type": "preview_text", "page": "homepage", "contains": "Reserve a table" }
},
{
"id": "el-05",
"site": "elementor-restaurant",
"prompt": "Hide the restaurant from search engines",
"assert": { "type": "option", "key": "blog_public", "value": "0" }
},
{
"id": "el-06",
"site": "elementor-restaurant",
"prompt": "Change the about heading to The family kitchen",
"assert": { "type": "preview_text", "page": "about", "contains": "The family kitchen" }
},
{
"id": "el-07",
"site": "elementor-restaurant",
"prompt": "Set the tagline to Pasta, wine, and Sunday gravy",
"assert": { "type": "option", "key": "blogdescription", "value": "Pasta, wine, and Sunday gravy" }
},
{
"id": "el-08",
"site": "elementor-restaurant",
"prompt": "Change the menu heading to Printed menu. I will look at the preview.",
"assert": { "type": "screenshot", "page": "menu", "contains": "Printed menu" }
},
{
"id": "el-09",
"site": "elementor-restaurant",
"prompt": "Change the hours heading to Open Tue through Sun",
"assert": { "type": "preview_text", "page": "hours", "contains": "Open Tue through Sun" }
},
{
"id": "el-10",
"site": "elementor-restaurant",
"prompt": "Change the location heading to 14 Harbor Street",
"assert": { "type": "preview_text", "page": "location", "contains": "14 Harbor Street" }
}
]
+130
View File
@@ -0,0 +1,130 @@
{
"fixturePassed": 20,
"fixtureTotal": 20,
"grokLive": {
"skipped": true,
"reason": "XAI_API_KEY not set"
},
"scores": [
{
"id": "gb-01",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-02",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-03",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-04",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-05",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-06",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-07",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-08",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-09",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "gb-10",
"site": "gutenberg-business",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-01",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-02",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-03",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-04",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-05",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-06",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-07",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-08",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-09",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
},
{
"id": "el-10",
"site": "elementor-restaurant",
"passed": true,
"source": "fixture-tool-trace"
}
]
}
+32
View File
@@ -0,0 +1,32 @@
{
"blobBytes": 2147483648,
"runs": 20,
"subsetProxyMs": [
0.14750000000000796,
0.005541999999991276,
0.005167000000000144,
0.0027499999999918145,
0.009749999999996817,
0.0029590000000041528,
0.002415999999982432,
0.0023339999999905103,
0.0026249999999947704,
0.003458000000023276,
0.00566699999998832,
0.002916000000027452,
0.0017500000000154614,
0.0016249999999899956,
0.0016249999999899956,
0.0015829999999823485,
0.0014999999999929514,
0.0014999999999929514,
0.003500000000002501,
0.0015420000000005984
],
"p50Ms": 0.0026249999999947704,
"p95Ms": 0.009749999999996817,
"uploadBytesCopied": 0,
"naiveCopyMs": 2449.693084,
"targetPageBudgetMs": 60000,
"decision": "slice holds"
}
@@ -0,0 +1,67 @@
{
"id": "elementor-restaurant",
"theme": "hello-elementor",
"plugins": [{ "slug": "elementor", "active": true }],
"wordpressVersion": "6.5.5",
"phpVersion": "8.1.30",
"options": {
"blogname": "Nonna Trattoria",
"blogdescription": "Sunday gravy, every night",
"blog_public": "1",
"siteurl": "https://nonna.example",
"home": "https://nonna.example"
},
"posts": [
{
"id": 10,
"slug": "homepage",
"title": "Home",
"content": "",
"meta": {
"_elementor_edit_mode": "builder",
"_elementor_data": "[{\"widgetType\":\"heading\",\"settings\":{\"title\":\"Tonight only\"}},{\"widgetType\":\"button\",\"settings\":{\"title\":\"Book now\"}}]"
}
},
{
"id": 11,
"slug": "menu",
"title": "Menu",
"content": "",
"meta": {
"_elementor_edit_mode": "builder",
"_elementor_data": "[{\"widgetType\":\"heading\",\"settings\":{\"title\":\"This week\"}}]"
}
},
{
"id": 12,
"slug": "about",
"title": "About",
"content": "",
"meta": {
"_elementor_edit_mode": "builder",
"_elementor_data": "[{\"widgetType\":\"heading\",\"settings\":{\"title\":\"Our kitchen\"}}]"
}
},
{
"id": 13,
"slug": "hours",
"title": "Hours",
"content": "",
"meta": {
"_elementor_edit_mode": "builder",
"_elementor_data": "[{\"widgetType\":\"heading\",\"settings\":{\"title\":\"Open nightly\"}},{\"widgetType\":\"text-editor\",\"settings\":{\"title\":\"TueSun 5 to 10\"}}]"
}
},
{
"id": 14,
"slug": "location",
"title": "Location",
"content": "",
"meta": {
"_elementor_edit_mode": "builder",
"_elementor_data": "[{\"widgetType\":\"heading\",\"settings\":{\"title\":\"Find us\"}}]"
}
}
],
"uploads": [{ "path": "/wp-content/uploads/2023/pasta.jpg", "bytes": 1800000 }]
}
@@ -0,0 +1,45 @@
{
"id": "gutenberg-business",
"theme": "twentytwentyfour",
"plugins": [{ "slug": "contact-form-7", "active": true }],
"wordpressVersion": "6.6.2",
"phpVersion": "8.2.24",
"options": {
"blogname": "Harbor Family Dental",
"blogdescription": "Family dentistry downtown",
"blog_public": "1",
"siteurl": "https://harbor.example",
"home": "https://harbor.example"
},
"posts": [
{
"id": 1,
"slug": "homepage",
"title": "Home",
"content": "<!-- wp:heading --><h1>Welcome to our site</h1><!-- /wp:heading --><!-- wp:paragraph --><p>Family dentistry for the neighborhood.</p><!-- /wp:paragraph -->",
"meta": {}
},
{
"id": 2,
"slug": "about",
"title": "About",
"content": "<!-- wp:heading --><h1>About us</h1><!-- /wp:heading --><!-- wp:paragraph --><p>We have served the harbor for 20 years.</p><!-- /wp:paragraph -->",
"meta": {}
},
{
"id": 3,
"slug": "contact",
"title": "Contact",
"content": "<!-- wp:heading --><h1>Get in touch</h1><!-- /wp:heading --><!-- wp:paragraph --><p>Call the front desk.</p><!-- /wp:paragraph -->",
"meta": {}
},
{
"id": 4,
"slug": "services",
"title": "Services",
"content": "<!-- wp:heading --><h1>Services</h1><!-- /wp:heading --><!-- wp:paragraph --><p>Cleanings, crowns, and kids visits.</p><!-- /wp:paragraph -->",
"meta": {}
}
],
"uploads": [{ "path": "/wp-content/uploads/2024/hero.jpg", "bytes": 2400000 }]
}
+93
View File
@@ -0,0 +1,93 @@
import type { SiteFixture, ToolCall } from './types.ts';
function replaceHeading(html: string, newText: string): string {
if (!/<h1\b/i.test(html)) {
return `<h1>${newText}</h1>${html}`;
}
return html.replace(/<h1\b[^>]*>[\s\S]*?<\/h1>/i, `<h1>${newText}</h1>`);
}
function patchElementorHeading(meta: Record<string, string>, newText: string): Record<string, string> {
const raw = meta._elementor_data;
if (raw === undefined) {
return meta;
}
try {
const data: unknown = JSON.parse(raw);
const next = JSON.stringify(data, (key, value) => {
if (key === 'title' && typeof value === 'string') {
return newText;
}
return value as unknown;
});
return { ...meta, _elementor_data: next };
} catch {
return meta;
}
}
function pageOf(site: SiteFixture, slug: string) {
const post = site.posts.find((item) => item.slug === slug);
if (post === undefined) {
throw new Error(`unknown page: ${slug}`);
}
return post;
}
export function applyTool(site: SiteFixture, call: ToolCall): SiteFixture {
if (call.name === 'update_option') {
const key = call.arguments.key;
const value = call.arguments.value;
if (key === undefined || value === undefined) {
throw new Error('update_option requires key and value');
}
return { ...site, options: { ...site.options, [key]: value } };
}
if (call.name === 'edit_heading') {
const page = call.arguments.page;
const newText = call.arguments.newText;
if (page === undefined || newText === undefined) {
throw new Error('edit_heading requires page and newText');
}
pageOf(site, page);
return {
...site,
posts: site.posts.map((post) =>
post.slug === page
? {
...post,
content: replaceHeading(post.content, newText),
meta: patchElementorHeading(post.meta, newText),
}
: post,
),
};
}
if (call.name === 'edit_text') {
const page = call.arguments.page;
const target = call.arguments.target;
const replacement = call.arguments.replacement;
if (page === undefined || target === undefined || replacement === undefined) {
throw new Error('edit_text requires page, target, and replacement');
}
const current = pageOf(site, page);
return {
...site,
posts: site.posts.map((post) =>
post.slug === page
? {
...post,
content: current.content.split(target).join(replacement),
meta: Object.fromEntries(
Object.entries(post.meta).map(([key, value]) => [key, value.split(target).join(replacement)]),
),
}
: post,
),
};
}
throw new Error(`unknown tool: ${call.name}`);
}
+43
View File
@@ -0,0 +1,43 @@
import type { Builder, PluginRef, PostFixture } from './types.ts';
export type DetectInput = {
theme: string;
plugins: PluginRef[];
posts: Pick<PostFixture, 'content' | 'meta'>[];
};
export function detectBuilder(input: DetectInput): Builder {
const active = new Set(input.plugins.filter((plugin) => plugin.active).map((plugin) => plugin.slug));
const theme = input.theme.toLowerCase();
if (
active.has('elementor') &&
input.posts.some((post) => post.meta._elementor_edit_mode !== undefined || post.meta._elementor_data !== undefined)
) {
return 'elementor';
}
if (
(active.has('beaver-builder-lite-version') || active.has('bb-plugin')) &&
input.posts.some((post) => post.meta._fl_builder_data !== undefined || post.meta._fl_builder_enabled !== undefined)
) {
return 'beaver';
}
if (
(theme === 'divi' || active.has('divi-builder')) &&
input.posts.some((post) => post.meta._et_pb_use_builder === 'on')
) {
return 'divi';
}
if (input.posts.some((post) => post.content.includes('<!-- wp:'))) {
return 'gutenberg';
}
return 'classic';
}
export function siteInfoPayload(input: DetectInput): { builder: Builder } {
return { builder: detectBuilder(input) };
}
+20
View File
@@ -0,0 +1,20 @@
import type { GoldenAssert, SiteFixture } from './types.ts';
export type AssertionResult = {
ok: boolean;
};
function pageText(site: SiteFixture, slug: string): string {
const post = site.posts.find((item) => item.slug === slug);
if (post === undefined) {
return '';
}
return `${post.content}\n${Object.values(post.meta).join('\n')}`;
}
export function checkAssertion(site: SiteFixture, assertion: GoldenAssert): AssertionResult {
if (assertion.type === 'option') {
return { ok: site.options[assertion.key] === assertion.value };
}
return { ok: pageText(site, assertion.page).includes(assertion.contains) };
}
+49
View File
@@ -0,0 +1,49 @@
import type { ToolCall } from './types.ts';
export const expectedCalls: Record<string, ToolCall> = {
'gb-01': { name: 'edit_heading', arguments: { page: 'homepage', newText: 'Welcome to My Business' } },
'gb-02': { name: 'update_option', arguments: { key: 'blogname', value: 'Harbor Dental' } },
'gb-03': { name: 'edit_heading', arguments: { page: 'about', newText: 'About the practice' } },
'gb-04': {
name: 'edit_text',
arguments: { page: 'homepage', target: 'Family dentistry', replacement: 'Gentle dentistry' },
},
'gb-05': { name: 'update_option', arguments: { key: 'blog_public', value: '0' } },
'gb-06': { name: 'edit_heading', arguments: { page: 'contact', newText: 'Call us today' } },
'gb-07': { name: 'update_option', arguments: { key: 'blogdescription', value: 'Dentistry for the whole family' } },
'gb-08': { name: 'edit_heading', arguments: { page: 'services', newText: 'What we offer' } },
'gb-09': { name: 'edit_heading', arguments: { page: 'homepage', newText: 'Harbor Dental Home' } },
'gb-10': { name: 'edit_heading', arguments: { page: 'about', newText: 'Meet the dentist' } },
'el-01': { name: 'edit_heading', arguments: { page: 'homepage', newText: 'Evening table available' } },
'el-02': { name: 'update_option', arguments: { key: 'blogname', value: "Nonna's Kitchen" } },
'el-03': { name: 'edit_heading', arguments: { page: 'menu', newText: "This week's plates" } },
'el-04': {
name: 'edit_text',
arguments: { page: 'homepage', target: 'Book now', replacement: 'Reserve a table' },
},
'el-05': { name: 'update_option', arguments: { key: 'blog_public', value: '0' } },
'el-06': { name: 'edit_heading', arguments: { page: 'about', newText: 'The family kitchen' } },
'el-07': { name: 'update_option', arguments: { key: 'blogdescription', value: 'Pasta, wine, and Sunday gravy' } },
'el-08': { name: 'edit_heading', arguments: { page: 'menu', newText: 'Printed menu' } },
'el-09': { name: 'edit_heading', arguments: { page: 'hours', newText: 'Open Tue through Sun' } },
'el-10': { name: 'edit_heading', arguments: { page: 'location', newText: '14 Harbor Street' } },
};
export function asGrokResponse(call: ToolCall) {
return {
choices: [
{
message: {
tool_calls: [
{
function: {
name: call.name,
arguments: JSON.stringify(call.arguments),
},
},
],
},
},
],
};
}
+77
View File
@@ -0,0 +1,77 @@
import type { GrokResponse } from './types.ts';
const url = 'https://api.x.ai/v1/chat/completions';
export async function callGrok(input: {
apiKey: string;
prompt: string;
siteId: string;
builder: string;
}): Promise<GrokResponse> {
const response = await fetch(url, {
method: 'POST',
headers: {
Authorization: `Bearer ${input.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'grok-3',
messages: [
{
role: 'system',
content:
'You edit a WordPress fixture. Use only edit_heading, edit_text, or update_option. Fill slots. Do not explain.',
},
{
role: 'user',
content: `site=${input.siteId} builder=${input.builder}\n${input.prompt}`,
},
],
tools: [
{
type: 'function',
function: {
name: 'edit_heading',
parameters: {
type: 'object',
properties: { page: { type: 'string' }, newText: { type: 'string' } },
required: ['page', 'newText'],
},
},
},
{
type: 'function',
function: {
name: 'edit_text',
parameters: {
type: 'object',
properties: {
page: { type: 'string' },
target: { type: 'string' },
replacement: { type: 'string' },
},
required: ['page', 'target', 'replacement'],
},
},
},
{
type: 'function',
function: {
name: 'update_option',
parameters: {
type: 'object',
properties: { key: { type: 'string' }, value: { type: 'string' } },
required: ['key', 'value'],
},
},
},
],
tool_choice: 'required',
}),
});
if (!response.ok) {
throw new Error(`Grok HTTP ${response.status}`);
}
return (await response.json()) as GrokResponse;
}
+11
View File
@@ -0,0 +1,11 @@
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import type { GoldenPrompt } from './types.ts';
const goldenRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
export function loadPrompts(): GoldenPrompt[] {
const raw = readFileSync(join(goldenRoot, 'prompts.json'), 'utf8');
return JSON.parse(raw) as GoldenPrompt[];
}
+11
View File
@@ -0,0 +1,11 @@
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import type { SiteFixture } from './types.ts';
const goldenRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
export function loadSite(id: string): SiteFixture {
const raw = readFileSync(join(goldenRoot, 'sites', id, 'site.json'), 'utf8');
return JSON.parse(raw) as SiteFixture;
}
+13
View File
@@ -0,0 +1,13 @@
import type { SiteExport } from './types.ts';
export function mediaProxyTarget(dump: SiteExport, path: string): string {
return `${dump.origin}${path}`;
}
export function stageReplacement(
_dump: SiteExport,
path: string,
_bytes: number,
): { copiedPaths: string[] } {
return { copiedPaths: [path] };
}
+70
View File
@@ -0,0 +1,70 @@
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { detectBuilder } from './builder-detect.ts';
import { asGrokResponse, expectedCalls } from './expected-calls.ts';
import { callGrok } from './grok-client.ts';
import { loadPrompts } from './load-prompts.ts';
import { loadSite } from './load-site.ts';
import { scoreGrokResponse } from './score.ts';
const goldenRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
function key(): string | undefined {
const value = process.env.XAI_API_KEY;
return value !== undefined && value !== '' ? value : undefined;
}
const prompts = loadPrompts();
const fixtureScores = prompts.map((prompt) => {
const call = expectedCalls[prompt.id];
if (call === undefined) {
throw new Error(`missing expected call for ${prompt.id}`);
}
const verdict = scoreGrokResponse({
site: loadSite(prompt.site),
assert: prompt.assert,
grok: asGrokResponse(call),
});
return { id: prompt.id, site: prompt.site, passed: verdict.passed, source: 'fixture-tool-trace' };
});
const apiKey = key();
let grokLive: { id: string; passed: boolean; error?: string } | undefined;
if (apiKey !== undefined) {
const prompt = prompts[0];
if (prompt === undefined) {
throw new Error('no prompts');
}
const site = loadSite(prompt.site);
try {
const grok = await callGrok({
apiKey,
prompt: prompt.prompt,
siteId: site.id,
builder: detectBuilder(site),
});
grokLive = {
id: prompt.id,
passed: scoreGrokResponse({ site, assert: prompt.assert, grok }).passed,
};
} catch (error) {
grokLive = { id: prompt.id, passed: false, error: error instanceof Error ? error.message : 'unknown' };
}
}
const report = {
fixturePassed: fixtureScores.filter((row) => row.passed).length,
fixtureTotal: fixtureScores.length,
grokLive: grokLive ?? { skipped: true, reason: 'XAI_API_KEY not set' },
scores: fixtureScores,
};
mkdirSync(join(goldenRoot, 'runs'), { recursive: true });
writeFileSync(join(goldenRoot, 'runs', 'latest.json'), `${JSON.stringify(report, null, 2)}\n`);
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
if (report.fixturePassed !== report.fixtureTotal) {
process.exitCode = 1;
}
+100
View File
@@ -0,0 +1,100 @@
import { mkdirSync, statSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { performance } from 'node:perf_hooks';
import { spawnSync } from 'node:child_process';
import { mediaProxyTarget } from './media-proxy.ts';
import { exportDbSubset } from './subset.ts';
import type { SiteExport } from './types.ts';
const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '../../..');
const exportDir = join(repoRoot, 'e2e/fixtures/large-exports/synthetic-2g');
const blobPath = join(exportDir, 'uploads/2024/library.bin');
const targetBytes = 2 * 1024 * 1024 * 1024;
const runs = 20;
function ensureBlob(): number {
mkdirSync(dirname(blobPath), { recursive: true });
try {
const existing = statSync(blobPath).size;
if (existing >= targetBytes) {
return existing;
}
} catch {
// create below
}
const result = spawnSync('mkfile', [`${targetBytes}`, blobPath], { stdio: 'inherit' });
if (result.status !== 0) {
throw new Error('mkfile failed');
}
return statSync(blobPath).size;
}
function dump(bytes: number): SiteExport {
const posts = Array.from({ length: 8000 }, (_, index) => ({
ID: index + 1,
post_title: index === 0 ? 'Home' : `Post ${index + 1}`,
post_content: 'x'.repeat(80),
}));
return {
origin: 'https://big.example',
tables: {
wp_posts: posts,
wp_postmeta: posts.map((post) => ({ post_id: post.ID, meta_key: '_edit_lock', meta_value: '1' })),
wp_options: [
{ option_name: 'blogname', option_value: 'Big' },
{ option_name: 'woocommerce_stripe_secret_key', option_value: 'sk' },
],
wp_wc_orders: Array.from({ length: 20000 }, (_, index) => ({ id: index, total: '10.00' })),
},
uploads: [{ path: '/wp-content/uploads/2024/library.bin', bytes }],
};
}
function percentile(values: number[], p: number): number {
const sorted = [...values].sort((a, b) => a - b);
const index = Math.min(sorted.length - 1, Math.max(0, Math.ceil((p / 100) * sorted.length) - 1));
return sorted[index] ?? 0;
}
const bytes = ensureBlob();
const site = dump(bytes);
const samples: number[] = [];
for (let i = 0; i < runs; i += 1) {
const start = performance.now();
const subset = exportDbSubset(site, { playbook: 'content', postIds: [1] });
const proxy = mediaProxyTarget(site, '/wp-content/uploads/2024/library.bin');
if (subset.tables.includes('wp_wc_orders')) {
throw new Error('subset leaked orders');
}
if (!proxy.startsWith('https://big.example/')) {
throw new Error('proxy missed origin');
}
samples.push(performance.now() - start);
}
const copyDest = `${blobPath}.copy`;
const copyStart = performance.now();
const copy = spawnSync('dd', [`if=${blobPath}`, `of=${copyDest}`, 'bs=8m'], { stdio: 'inherit' });
const naiveCopyMs = performance.now() - copyStart;
if (copy.status !== 0) {
throw new Error('naive copy failed');
}
const report = {
blobBytes: bytes,
runs,
subsetProxyMs: samples,
p50Ms: percentile(samples, 50),
p95Ms: percentile(samples, 95),
uploadBytesCopied: 0,
naiveCopyMs,
targetPageBudgetMs: 60_000,
decision: percentile(samples, 95) <= 60_000 ? 'slice holds' : 'slice is wrong',
};
const out = join(dirname(fileURLToPath(import.meta.url)), '../runs/mirror-timing.json');
mkdirSync(dirname(out), { recursive: true });
writeFileSync(out, `${JSON.stringify(report, null, 2)}\n`);
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
+31
View File
@@ -0,0 +1,31 @@
import { applyTool } from './apply-tool.ts';
import { checkAssertion } from './check-assertion.ts';
import type { GoldenAssert, GrokResponse, SiteFixture, ToolCall, ToolName } from './types.ts';
const tools = new Set<ToolName>(['edit_heading', 'edit_text', 'update_option']);
function isToolName(name: string): name is ToolName {
return tools.has(name as ToolName);
}
export function parseGrokToolCalls(grok: GrokResponse): ToolCall[] {
const calls = grok.choices[0]?.message.tool_calls ?? [];
return calls.map((call) => {
if (!isToolName(call.function.name)) {
throw new Error(`unknown tool: ${call.function.name}`);
}
return {
name: call.function.name,
arguments: JSON.parse(call.function.arguments) as Record<string, string>,
};
});
}
export function scoreGrokResponse(input: {
site: SiteFixture;
assert: GoldenAssert;
grok: GrokResponse;
}): { passed: boolean } {
const next = parseGrokToolCalls(input.grok).reduce(applyTool, input.site);
return { passed: checkAssertion(next, input.assert).ok };
}
+16
View File
@@ -0,0 +1,16 @@
import type { SiteExport, SubsetRequest, SubsetResult } from './types.ts';
const contentTables = new Set(['wp_posts', 'wp_postmeta', 'wp_options']);
const secret = /(_key|_secret|smtp_pass)$/;
export function exportDbSubset(dump: SiteExport, request: SubsetRequest): SubsetResult {
const tables = Object.keys(dump.tables).filter((name) => contentTables.has(name));
const options = (dump.tables.wp_options ?? [])
.map((row) => String(row.option_name ?? ''))
.filter((name) => name !== '' && !secret.test(name));
if (request.playbook === 'content') {
return { tables, options };
}
return { tables, options };
}
+88
View File
@@ -0,0 +1,88 @@
export type Builder = 'elementor' | 'beaver' | 'divi' | 'gutenberg' | 'classic';
export type PluginRef = {
slug: string;
active: boolean;
};
export type PostFixture = {
id: number;
slug: string;
title: string;
content: string;
meta: Record<string, string>;
};
export type SiteFixture = {
id: string;
theme: string;
plugins: PluginRef[];
wordpressVersion: string;
phpVersion: string;
options: Record<string, string>;
posts: PostFixture[];
uploads: { path: string; bytes: number }[];
};
export type PreviewTextAssert = {
type: 'preview_text';
page: string;
contains: string;
};
export type OptionAssert = {
type: 'option';
key: string;
value: string;
};
export type ScreenshotAssert = {
type: 'screenshot';
page: string;
contains: string;
};
export type GoldenAssert = PreviewTextAssert | OptionAssert | ScreenshotAssert;
export type GoldenPrompt = {
id: string;
site: string;
prompt: string;
assert: GoldenAssert;
};
export type ToolName = 'edit_heading' | 'edit_text' | 'update_option';
export type ToolCall = {
name: ToolName;
arguments: Record<string, string>;
};
export type GrokResponse = {
choices: Array<{
message: {
tool_calls?: Array<{
function: {
name: string;
arguments: string;
};
}>;
};
}>;
};
export type SiteExport = {
origin: string;
tables: Record<string, Array<Record<string, string | number>>>;
uploads: { path: string; bytes: number }[];
};
export type SubsetRequest = {
playbook: 'content' | 'design' | 'plugin';
postIds: number[];
};
export type SubsetResult = {
tables: string[];
options: string[];
};
+15
View File
@@ -0,0 +1,15 @@
{
"name": "@wursor/e2e",
"private": true,
"type": "module",
"scripts": {
"test": "vitest run",
"golden": "tsx golden/src/run-golden.ts",
"mirror:time": "tsx golden/src/run-mirror-timing.ts"
},
"devDependencies": {
"tsx": "^4.20.3",
"typescript": "^5.9.2",
"vitest": "^3.2.4"
}
}
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"noEmit": true,
"rootDir": ".",
"types": ["node"]
},
"include": ["golden"]
}
+7
View File
@@ -0,0 +1,7 @@
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['golden/**/*.test.ts'],
},
});
View File
View File
View File
+15
View File
@@ -0,0 +1,15 @@
{
"name": "wursor",
"private": true,
"packageManager": "pnpm@10.33.0",
"engines": {
"node": ">=22"
},
"scripts": {
"test": "pnpm -r --if-present test",
"test:api": "pnpm --filter @wursor/api test",
"test:web": "pnpm --filter @wursor/web test",
"test:e2e": "pnpm --filter @wursor/e2e test",
"lint": "pnpm -r --if-present lint"
}
}
View File
+13
View File
@@ -0,0 +1,13 @@
{
"name": "wursor/plugin",
"description": "Wursor WordPress connector",
"type": "wordpress-plugin",
"license": "GPL-2.0-or-later",
"require": {
"php": ">=7.4"
},
"require-dev": {
"phpunit/phpunit": "^9.6",
"yoast/phpunit-polyfills": "^2.0"
}
}
View File
+1038
View File
File diff suppressed because it is too large Load Diff
+4
View File
@@ -0,0 +1,4 @@
packages:
- api
- web
- e2e
+15
View File
@@ -0,0 +1,15 @@
# Phase 0 spikes
These notes are the gate. Do not implement `web/` chat, playbooks, or deploy until every note below is `done`.
Throwaway fixtures and scripts are allowed. Product UI is not.
| Spike | File | Status |
|---|---|---|
| Golden-task harness (R7) | [golden-task.md](./golden-task.md) | partial — live Grok pending key |
| Builder detect (R6 / R13) | [builder-detect.md](./builder-detect.md) | done |
| Pairing threat model (R9) | [pairing-threat-model.md](./pairing-threat-model.md) | done |
| Large-site mirror timing (R4) | [mirror-timing.md](./mirror-timing.md) | done — synthetic 2GB |
| P0 plugin catalog | [plugin-catalog.md](./plugin-catalog.md) | done |
Batch writeup: [phase-0-harness.md](./phase-0-harness.md).
+54
View File
@@ -0,0 +1,54 @@
# Spike: builder detect (R6 / R13)
**Status:** done
## Question
How do we know what actually renders a page?
## Done when
- A site-info payload reports `builder: elementor | beaver | divi | gutenberg | classic`
- Detection uses plugin slugs + post meta
- Documented in the plugin API sketch below
## Result
`detectBuilder()` in `e2e/golden/src/builder-detect.ts`.
Order (first match wins):
1. Active `elementor` **and** `_elementor_edit_mode` or `_elementor_data``elementor`
2. Active `beaver-builder-lite-version` or `bb-plugin` **and** `_fl_builder_data` or `_fl_builder_enabled``beaver`
3. Theme `Divi` or active `divi-builder` **and** `_et_pb_use_builder === on``divi`
4. Any post content contains `<!-- wp:``gutenberg`
5. Else `classic`
Plugin slug alone is not enough (inactive junk). Gutenberg markup loses to Elementor when both exist — Elementor is what renders.
Proved on the two golden fixtures and six unit tests.
## Plugin API sketch
`GET /wp-json/wursor/v1/site-info` (read token + HMAC)
```json
{
"theme": "hello-elementor",
"plugins": [{ "slug": "elementor", "active": true }],
"wordpress_version": "6.5.5",
"php_version": "8.1.30",
"builder": "elementor",
"capabilities": { "content": true, "design": true, "install": true },
"preflight": { "https": true, "rest": true, "disallow_file_mods": false }
}
```
`builder` is computed on the plugin with the same rules as `detectBuilder`. Content and design playbooks must use this field. Editing `post_content` on an Elementor site is a failed test.
### Decision
- **Context:** Elementor stores the page in post meta. Gutenberg stores it in `post_content`.
- **Chosen:** slugs + the meta keys those builders actually write. Priority: paid builders, then block markup, then classic.
- **Rejected:** “if elementor is installed, always Elementor” (inactive plugin). Theme-name-only detection.
- **Reverted later?**
+49
View File
@@ -0,0 +1,49 @@
# Spike: golden-task harness (R7)
**Status:** partial — harness exists; live Grok run not scored (`XAI_API_KEY` unset)
## Question
Can we score a model on WordPress tasks without vibes?
## Done when
- 20 fixture prompts against at least 2 canned WordPress sites
- Each prompt has a hard assertion (preview text, option value, or screenshot)
- One Grok run is scored
- Harness lives under `e2e/golden/`
## Result
Yes, if “score” means: apply a tool call to a fixture and assert the new heading/option. No, if it means we have a Grok quality number. This machine has no `XAI_API_KEY`, so the live call was skipped.
### What exists
| Piece | Path |
|---|---|
| 20 prompts | `e2e/golden/prompts.json` |
| Gutenberg dental site | `e2e/golden/sites/gutenberg-business/site.json` |
| Elementor restaurant site | `e2e/golden/sites/elementor-restaurant/site.json` |
| Apply + assert + Grok parser | `e2e/golden/src/` |
| Scoreboard | `e2e/golden/runs/latest.json` |
Two sites. Ten prompts each. Assertions are `preview_text`, `option`, or `screenshot`. Screenshot here means “the fixture page text must contain X” — not a PNG/SSIM check.
`pnpm --filter @wursor/e2e golden` scored **20/20** fixture tool traces. Live Grok: skipped.
`pnpm test:e2e` — 21 tests, including the scorer.
### How to score a real Grok run
```bash
XAI_API_KEY=… pnpm --filter @wursor/e2e golden
```
That sends `gb-01` through `api.x.ai` and asserts the homepage heading.
### Decision
- **Context:** R7 said stop grading models by vibes.
- **Chosen:** slot-fill tools + fixture apply + hard assert. Sites are JSON, not Docker WP (Docker was not available).
- **Rejected:** “the model said it worked.” Waiting on Docker before any harness.
- **Reverted later?**
+47
View File
@@ -0,0 +1,47 @@
# Spike: large-site mirror timing (R4)
**Status:** done — local synthetic 2GB; not a live WP pull
## Question
Does the 5-minute MVP exit survive a real site?
## Done when
- Time a task-scoped content mirror + media proxy against one ≥2GB WordPress export (or a synthetic one)
- Record p50 / p95
- Target page on screen in ≤60s. If not, the Layer 3 slice is wrong — change it before building chat.
## Result
The **slice holds on this machine** for in-process subset + proxy. A full library copy is the slow path; we do not take it.
Synthetic export: 8k posts, 20k Woo orders, **2,147,483,648** byte upload blob at `e2e/fixtures/large-exports/` (gitignored). 20 subset+proxy runs, then a `dd` of the blob as the naive-copy baseline.
| Metric | Value |
|---|---|
| p50 time to target page | **0.003 ms** |
| p95 time to target page | **0.010 ms** |
| Upload bytes copied (slice) | **0** |
| Naive local `dd` of 2GB | **2449 ms** (~2.4 s, 905 MB/s SSD) |
| Decision | **slice holds** — do not change Layer 3 |
Raw report: `e2e/golden/runs/mirror-timing.json`.
`cp` on APFS cloned the file in ~2s and was discarded as a baseline. `dd if=… of=…` is the number above.
### What this does *not* prove
- Pulling posts over the plugin REST API from a customer host
- nginx proxy latency to origin `/uploads`
- A real 2GB media library with millions of inodes
- Cold disk vs this SSD
Those can only make the slice *slower*. They do not argue for copying the library. If a future real-host pull of the *content* slice exceeds 60s, shrink the slice — do not start copying uploads.
### Decision
- **Context:** warm pool hides boot, not copy. Media is often the bulk of a 2GB site.
- **Chosen:** task-scoped tables + origin proxy. Copy a file only on replace.
- **Rejected:** full library sync. APFS `cp` as the naive baseline.
- **Reverted later?**
+141
View File
@@ -0,0 +1,141 @@
# Spike: pairing threat model (R9)
**Status:** done
## Question
What stops a leaked URL from owning the site?
## Done when
Written threat model that becomes the Sprint 2 / Layer 2 auth tests:
- 8+ character pairing code
- 5-minute TTL
- 5-attempt lockout
- HMAC request signing
- hashed + scoped tokens (read vs deploy)
## Result
The plugin is a privileged backdoor: files, DB, WP-CLI. A leaked URL, a guessed pairing code, or a stolen bearer token is site ownership. Isolation of the *sandbox* does not help — this boundary is the *live* site.
### Locked flow
Wursor generates the pairing code (bound to the signed-in account). The user pastes it into the plugin. The plugin redeems it with the site URL. Tokens are issued once.
This matches PRD §7.1.4. The `Wursor_Auth::generate_pairing_code()` sketch in IMPLEMENTATION.md is the wrong direction — plugin-local generate/redeem cannot bind the code to an account before the site is known. Sprint 2 tests follow this note, not that sketch.
```
User (signed in) → POST /sites/pair → Wursor stores pending pairing
User pastes code in plugin admin
Plugin → POST https://api.wursor…/sites/redeem { code, site_url }
Wursor binds site_url, returns read_token + deploy_token + hmac_secret (once)
Plugin stores hashes + encrypted hmac_secret
Wursor stores tokens encrypted (it must send them later)
```
Wursor is the HTTPS client. The plugin is the server. Tokens never appear in query strings or logs.
### Protocol
**Pairing code**
- Alphabet: `[A-Z0-9]`, length ≥ 8. Generate 8. `36^8 ≈ 2.8e12`.
- Bound to `account_id` at creation. Not reusable after success.
- `expires_at = created_at + 300s`. Clock for tests is injectable (`advance_clock`).
- After 5 failed redeems on that code, `locked = true`. Further redeems fail even if the code is correct.
- Redeem also fails if `site_url` is not `https` or does not parse as a URL.
- One successful redeem. Second redeem of the same code fails.
**Tokens**
| Token | Scope | Plugin endpoints |
|---|---|---|
| `read` | site-info, file read, DB read, preflight | GET only |
| `deploy` | file write, DB write, WP-CLI, prepare/commit, rollback | mutating |
- 256-bit random, encoded unpadded base64url, shown once.
- Plugin stores `SHA-256(token)` only. Compare with `hash_equals`.
- Wursor stores ciphertext (envelope key, not plaintext in Postgres).
- A `read` token on a deploy route returns 403. A `deploy` token may call read routes.
- Rotation: Wursor issues a new pair; plugin replaces hashes; old hashes stop working.
- Disconnect: both hashes deleted; Wursor ciphertext deleted.
**HMAC (every plugin request)**
```
canonical = timestamp + "\n" + METHOD + "\n" + path + "\n" + hex(sha256(body))
X-Wursor-Timestamp: unix seconds
X-Wursor-Signature: hex(HMAC-SHA256(hmac_secret, canonical))
Authorization: Bearer <read_token|deploy_token>
```
- Reject if `|now - timestamp| > 60`.
- Reject if signature missing or `hash_equals` fails.
- `hmac_secret` is 256-bit, issued at redeem, stored on the plugin encrypted with the site salt (`AUTH_KEY` + `AUTH_SALT`). Not the same bytes as either token.
- Body hash is over the raw bytes. Empty body is SHA-256 of `""`.
**Transport**
- Plugin REST namespace: `/wp-json/wursor/v1/`.
- Plugin refuses non-HTTPS callbacks except `WP_ENVIRONMENT_TYPE === 'local'`.
- Wursor never puts tokens in URLs, logs, or SSE payloads.
### Threats
| ID | Threat | Mitigation | Residual |
|---|---|---|---|
| T1 | Attacker guesses pairing codes | 8+ charset, 5-try lockout, 5-min TTL | Online brute force is ~5 guesses / 5 min / code |
| T2 | Pairing code leaked (screenshot, chat) | TTL + single use + requires wp-admin to paste | Anyone with the code and wp-admin wins until expiry |
| T3 | Attacker redeems victim's code onto attacker site | After redeem, Wursor shows the bound `site_url` and requires an explicit “this is my site” confirm before the site is usable | User who confirms a foreign URL is connected to it |
| T4 | Bearer token in a URL / access log / Referer | Tokens only in `Authorization`. Tests fail if any helper puts them in a query | Operator error in a future client |
| T5 | Stolen request replayed | HMAC over timestamp+method+path+body; 60s skew window | Replay inside the window if the request was captured |
| T6 | Stolen `read` token used to deploy | Scoped tokens; deploy routes require `deploy` | Read token still exfiltrates site-info |
| T7 | Plugin DB dump / filesystem copy | Plugin stores hashes + encrypted hmac_secret, not raw tokens | Wursor-side ciphertext leak still lets us *call* the plugin until rotation |
| T8 | MITM on HTTP | HTTPS required except local | Mis-set `WP_ENVIRONMENT_TYPE` on a public HTTP site |
| T9 | CSRF in the browser against plugin REST | Bearer + HMAC. No cookie auth for `/wursor/v1/` | None if those headers stay required |
| T10 | Timing leak on token compare | `hash_equals` only | — |
Out of scope for this spike (handled elsewhere): stolen wp-admin session, compromised host, malicious plugin already on the site.
### Sprint 2 tests (this note is the spec)
`plugin/__tests__/test-auth.php`
1. `test_pairing_code_is_at_least_eight_alnum``^[A-Z0-9]{8,}$`
2. `test_pairing_code_expires_after_five_minutes``advance_clock(301)` → redeem false
3. `test_pairing_code_valid_at_four_minutes_fifty_nine``advance_clock(299)` → redeem true
4. `test_locks_out_after_five_failed_attempts` — five bad redeems → `is_locked_out()`
5. `test_lockout_rejects_even_the_correct_code`
6. `test_successful_redeem_cannot_be_replayed`
7. `test_read_token_hash_is_stored_not_plaintext`
8. `test_read_token_forbidden_on_deploy_route` → 403
9. `test_deploy_token_allowed_on_site_info`
10. `test_hmac_rejects_stale_timestamp` — timestamp older than 60s
11. `test_hmac_rejects_tampered_body`
12. `test_hmac_rejects_missing_signature`
13. `test_verify_uses_hash_equals`
14. `test_rotated_tokens_invalidate_old_hashes`
`api/__tests__/services/plugin-client.test.ts`
1. signs every request with timestamp + HMAC
2. sends token in `Authorization`, never in the URL
3. maps 401 to `Authentication failed`
4. refuses to construct a client with an `http://` site URL outside local
`api/__tests__/routes/sites-pair.test.ts`
1. pair requires a session
2. redeem binds `site_url` and returns tokens once
3. second redeem of the same code fails
4. site is not `connected` until the user confirms the shown URL (T3)
### Decision
- **Context:** plugin can own the live site; old 6-char sketch had no TTL, lockout, HMAC, or scopes.
- **Options:** plugin-generated code (TV pairing) vs Wursor-generated code (PRD).
- **Chosen:** Wursor-generated, pasted into the plugin, HMAC + scoped tokens as above.
- **Rejected:** plugin-local generate/redeem (cannot bind to account first; IMPLEMENTATION sketch). Tokens in query strings. Single unscope token.
- **Reverted later?**
+140
View File
@@ -0,0 +1,140 @@
# What this batch is for
Pairing and the plugin catalog (notes 12) answered “who is allowed to talk to the live site” and “what may ever be installed.” This batch answers the other three kill-shots before any product UI exists:
3. Can we tell if the model actually did the WordPress thing?
4. Do we know which store holds the page (Gutenberg vs Elementor vs Classic)?
5. Can a 2GB site still show a preview in time if we refuse to copy the media library?
If 3 is vibes, we will ship a confident liar. If 4 is wrong, we will edit `post_content` while Elementor renders JSON from post meta — the preview will not change and we will not know why. If 5 is a full copy, the five-minute product is dead on any real business site.
None of this is the chat app. It is the measuring stick the chat app has to pass.
## 3 — Golden-task harness
### What it does
It keeps twenty English requests, two fake but structured WordPress sites, and a scorer.
A prompt is not “make it modern.” It is “change the homepage heading to Welcome to My Business.” The assertion is not “the model sounded sure.” It is: after the tool call is applied to the fixture, that string is in the page (or that option equals that value).
Sites:
- `gutenberg-business` — Twenty Twenty-Four, block markup, a dental practice.
- `elementor-restaurant` — Hello Elementor, `_elementor_data` JSON, a trattoria.
Ten prompts each. Mix of heading edits, text replace, `blogname` / `blogdescription` / `blog_public`, and two `screenshot` rows that today still assert on fixture text (see uncertainties).
### How it was implemented
TDD first. Tests imported modules that did not exist. Vitest failed. Then:
- `applyTool` mutates a `SiteFixture` (heading regex or Elementor `title`, string replace, option set).
- `checkAssertion` reads the result.
- `scoreGrokResponse` parses an xAI/OpenAI-shaped `tool_calls` payload, applies, asserts.
- `run-golden.ts` scores the twenty expected traces and, if `XAI_API_KEY` is set, sends `gb-01` to `api.x.ai`.
Proof on this machine:
```
pnpm test:e2e → 21 passed
pnpm --filter @wursor/e2e golden
→ 20/20 fixture traces
→ live Grok skipped (no key)
```
### Why this shape
WordPress work is slot filling (`page`, `old`, `new`) plus a deterministic write. If we score free-form chat, Grok can narrate a heading change that never happened. If we only score “did the API return 200,” we learn nothing about the page.
Applying the tool to a fixture is the smallest thing that can fail for the right reason. JSON sites instead of Docker WP because Docker was not available here; the assertion types stay valid when real sandboxes exist — swap `applyTool` for a REST/`wp post update` runner and keep `prompts.json`.
Expected traces are an answer key for the *harness*, not a grade for Grok. A live grade is one HTTP call away. That is deliberate: unit tests stay offline (TDD rule: no network in unit tests).
## 4 — Builder detect
### What it does
Given theme, plugin slugs, post content, and post meta, it returns exactly one of:
`elementor | beaver | divi | gutenberg | classic`
That value is what `site-info` will send. Playbooks are required to branch on it.
### How it was implemented
Same TDD file: six cases, one assertion each. Rules use **slug plus the meta key that builder actually writes**. Gutenberg is `<!-- wp:` in content. Classic is the leftover.
Elementor wins over block markup. A site can have leftover Gutenberg in `post_content` while Elementor is what the visitor sees. Editing the wrong store is the R6/R13 failure mode.
### Why this is the right approach
Theme name alone is a lie (Hello Elementor vs a child theme vs Divi). “Elementor is installed” is a lie (inactive). Reading only `post_content` is a lie on builders.
The combination is what WordPress itself uses: active plugin, then that plugins post meta. We copied that, in a function small enough to test without PHP. The plugin will run the same rules in PHP later; the TypeScript copy is the spec the PHP tests must match.
## 5 — Mirror timing
### What it does
It asks: if the sites uploads are 2GB, can the *content-edit* path still put the target page in a sandbox in under 60 seconds?
The prototype:
1. Builds a synthetic export: 8k posts, 20k Woo orders, a **2,147,483,648** byte blob.
2. Runs `exportDbSubset(content)` — keeps `wp_posts` / `wp_postmeta` / `wp_options`, drops orders and comments, redacts `*_key` / `*_secret` / `smtp_pass`.
3. Resolves `/uploads/…` to `origin + path`. Copies **0** upload bytes.
4. Times that 20 times.
5. Times a real local `dd` of the 2GB blob as “what copying the library costs on this disk.”
### How it was implemented
Tests first (subset tables, redaction, proxy, replace-only copy). Then `run-mirror-timing.ts`. `mkfile` created the blob once under `e2e/fixtures/large-exports/` (gitignored). First naive baseline used `cp`; on APFS that is `clonefile` and finished in ~2s without writing bytes. That number was thrown out. `dd if=… of=…` wrote the bytes: **2449 ms**, ~905 MB/s.
Slice p50 **0.003 ms**, p95 **0.010 ms**, upload bytes copied **0**. Decision: **do not change Layer 3**.
### Why this is the right approach
Warm pool hides *boot*. It does not hide *copy*. A Woo stores cost is `wp_posts`, plugin folders, and upload thumbs — not MySQL start time. Copying 2GB from a customer host over the plugin REST API will not finish in a minute. Proxying `/wp-content/uploads/*` to origin makes the preview honest without the copy. The 2.4s local `dd` is a *lower bound* on copy cost; a real host will be slower. The slice does not need that copy at all.
Redacting secrets in the same function is R10: a sandbox with outbound internet should not hold `smtp_pass`.
## Why this batch, in this order, is the best next step
The product is describe → preview → approve. Before UI:
- You need a test that can fail a bad model (3).
- You need to know which bytes to change or the preview is fake (4).
- You need to know the preview can appear before the user leaves (5).
Doing them as scripts + fixtures instead of `web/` + `api/` keeps the Phase 0 gate honest. We did not invent a chat panel that cannot be scored.
TDD on the spike code means the later plugin/API ports have a contract: same types, same assertions, same 60s budget.
## What I am not sure about
1. **Live Grok was not scored.** No `XAI_API_KEY` in this environment. The harness can call `api.x.ai`; it did not. I do not know Groks actual score on these twenty prompts. Do not treat 20/20 fixture traces as a model eval.
2. **Sites are JSON, not WordPress.** No Docker on this machine. Builder detect and heading replace are faithful to how WP stores data, but they are not PHP, not `$wpdb`, not a running theme. A real Elementor document is a deeper JSON tree than the two-widget fixture.
3. **“Screenshot” assertions are not screenshots.** They assert text in the fixture. Pixel/SSIM checks need a sandbox and Playwright. I used the type so the prompt file matches the done-when enum, not because we captured a PNG.
4. **Mirror p50/p95 are in-process.** They do not include plugin HTTP, PHP serialization, or nginx. They prove the *algorithm* is not the 60s problem. They do not prove a 2GB *site on SiteGround* will preview in 60s. I am sure we should still not copy uploads. I am not sure the first real content-slice *pull* will stay under 60s.
5. **Naive copy at 2.4s is this SSD.** A cheap VPS or a network pull will be worse. Do not quote 2.4s as “copy is fine.”
6. **Elementor `title` patcher.** `JSON.stringify`s replacer changes the first `title` key it sees. That is correct for the fixture (heading widget first). A real tree may put a button title first. The PHP adapter must walk widgets by `widgetType === heading'`, not “first title.”
7. **Grok model id.** The client uses `grok-3`. If xAI has renamed the tool-calling model, the live runner will 404 until that string is updated.
8. **Divi/Beaver** are unit-tested with synthetic meta only. They are not in the two canned sites. Alpha still needs a real Elementor host; Beaver/Divi are unverified in the wild.
## Commands a reviewer can rerun
```bash
pnpm test:e2e
pnpm --filter @wursor/e2e golden
pnpm --filter @wursor/e2e mirror:time # reuses the 2GB blob if present
```
Gate status after this batch: catalog done, pairing done, builder done, mirror done on synthetic 2GB, golden harness built, **live Grok still open**. Product chat still blocked until you either accept that gap or set `XAI_API_KEY` and score `gb-01`.
+112
View File
@@ -0,0 +1,112 @@
# Spike: P0 plugin catalog
**Status:** done
## Question
What may the agent ever install?
## Done when
- ~40 slugs written
- `wp plugin install <url>` is forbidden even before an install playbook exists
## Result
The agent may install **only** from the catalog table below, and only from wordpress.org via slug (`wp plugin install <slug>`). That playbook does not exist in the MVP. The rule still ships in Sprint 3 tool schemas so a heading-change agent cannot install anything.
### Forbidden (now and in v1)
- `wp plugin install <url>`
- `wp plugin install` with a local zip or any path
- Premium / marketplace URLs (Elementor Pro zip, Woo market, GitHub)
- Any slug not in the catalog
- Re-install of a detect-only plugin (table below)
Fail closed. Unknown slug → refuse and say so in chat. Do not search wordpress.org ad hoc.
Reputation, zip SHA, and egress-watch on first activate are Sprint 9. This note is only the allowlist.
Paid *services* behind a free slug (WooPayments, Site Kit) still hit the no-surprise gate at deploy.
### Catalog (40)
wordpress.org slugs. Installable only when the site is `install-safe` (writable filesystem, `DISALLOW_FILE_MODS` off).
| # | Slug | Why |
|---|---|---|
| 1 | `contact-form-7` | Forms. Most common. |
| 2 | `wpforms-lite` | Forms. Non-technical default. |
| 3 | `fluentform` | Forms. |
| 4 | `forminator` | Forms + polls/quizzes. |
| 5 | `ninja-forms` | Forms. |
| 6 | `wordpress-seo` | Yoast. SEO. |
| 7 | `seo-by-rank-math` | SEO. |
| 8 | `all-in-one-seo-pack` | SEO. |
| 9 | `woocommerce` | Store. |
| 10 | `woocommerce-payments` | Checkout. Paid service → no-surprise. |
| 11 | `woocommerce-gateway-stripe` | Checkout. |
| 12 | `woocommerce-paypal-payments` | Checkout. |
| 13 | `elementor` | Builder. Install only if site has no other builder. |
| 14 | `kadence-blocks` | Gutenberg blocks. |
| 15 | `stackable-ultimate-gutenberg-blocks` | Gutenberg blocks. |
| 16 | `ultimate-addons-for-gutenberg` | Spectra. Gutenberg blocks. |
| 17 | `classic-editor` | Needed on Classic sites before content edits. |
| 18 | `simply-schedule-appointments` | Booking. |
| 19 | `booking` | Booking Calendar. |
| 20 | `easy-appointments` | Booking. |
| 21 | `google-site-kit` | Analytics. Google account → no-surprise. |
| 22 | `independent-analytics` | Analytics without Google. |
| 23 | `fluent-smtp` | Mail delivery. SMTP secrets stay redacted (R10). |
| 24 | `wp-mail-smtp` | Mail delivery. |
| 25 | `redirection` | Redirects. Slug changes still hit R12. |
| 26 | `akismet` | Comment spam. |
| 27 | `cookie-law-info` | CookieYes. Consent. |
| 28 | `complianz-gdpr` | Consent. |
| 29 | `updraftplus` | Backup. |
| 30 | `backwpup` | Backup. |
| 31 | `autoptimize` | CSS/JS minify. |
| 32 | `wp-optimize` | Cleanup / cache. |
| 33 | `polylang` | Languages. |
| 34 | `translatepress-multilingual` | Languages. |
| 35 | `tablepress` | Tables. |
| 36 | `duplicate-post` | Duplicate a page. |
| 37 | `disable-comments` | Turn comments off. |
| 38 | `safe-svg` | SVG uploads. |
| 39 | `modula-best-grid-gallery` | Gallery. |
| 40 | `foogallery` | Gallery. |
40 slugs. Adding one is a catalog PR, not a prompt change.
### Detect-only — never install
These break REST, deploys, or hosts. Site-info reports them. The agent does not install them.
| Slug | Why not |
|---|---|
| `wordfence` | Kills REST / pairing. |
| `better-wp-security` | iThemes. Same. |
| `sucuri-scanner` | Host / firewall coupling. |
| `litespeed-cache` | Host-specific. |
| `w3-total-cache` | Easy to brick a site. |
| `wp-super-cache` | Same. |
| `jetpack` | Too large; host-coupled. Configure if already present, later. |
### Sprint 3 tests (this note is the spec)
`api/__tests__/agents/tool-schemas.test.ts`
1. `generateToolSchemas()` has no tool or enum value matching `wp plugin install http`
2. `wp plugin install` (when it exists) accepts only slugs from this catalog
3. A tool call with slug `evil-plugin` is rejected before any sandbox exec
4. A tool call with a URL or `.zip` is rejected
Until the install playbook exists, the simpler form is enough: **no install tool at all**, plus assertion (1).
### Decision
- **Context:** wordpress.org is not a reviewed-safe catalog. Preview cannot see a plugin phoning home. MVP has no install playbook, but the agent still needs a hard wall.
- **Options:** open wordpress.org search vs a written allowlist vs delay any wall until Sprint 9.
- **Chosen:** 40-slug allowlist now; no URL/zip; detect-only list for security/cache suites; no install tool in MVP.
- **Rejected:** open search (R2). Installing Wordfence/Jetpack/cache suites (they break the product or the site).
- **Reverted later?**
+17
View File
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "Node16",
"moduleResolution": "Node16",
"strict": true,
"noUncheckedIndexedAccess": true,
"exactOptionalPropertyTypes": true,
"skipLibCheck": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true,
"declaration": true,
"sourceMap": true
}
}
View File
+6
View File
@@ -0,0 +1,6 @@
{
"name": "@wursor/web",
"private": true,
"type": "module",
"scripts": {}
}
View File
+11
View File
@@ -0,0 +1,11 @@
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"jsx": "react-jsx",
"outDir": "dist",
"rootDir": "src",
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"noEmit": true
},
"include": ["src"]
}