Add QA Test Strategy page — full ecosystem test plan with agent tagging

Tags: @Ody @OC @PI @hermes-contabo

47 test cases across 7 tiers covering:
- Hermes Plugin core + API
- Ody & PI agent clones
- OpenClaw provider
- Falah OS CE + Docker
- CI/CD pipeline
- E2E integration scenarios
- Pre-deploy smoke suite

Home.md

@@ -4,6 +4,7 @@ Welcome to the Hermes cPanel Agent wiki — documentation for running AI agent c
 
 ## Pages
 
+- **[QA Test Strategy](QA-Test-Strategy)** — Comprehensive QA plan for the entire Falah OS ecosystem
 - **[Cloudflare WAF Bypass Discovery](Cloudflare-WAF-Bypass-Discovery)** — How we discovered Cloudflare wasn't actually blocking the WP REST API
 - **[Getting Started](Getting-Started)** — Setup guide for new developers
 - **[Gitea Actions CI/CD](Gitea-Actions-CICD)** — Full CI/CD pipeline documentation
@@ -20,3 +21,4 @@ Welcome to the Hermes cPanel Agent wiki — documentation for running AI agent c
 
 - Code: https://git.falahos.my/wmj/hermes-cpanel-agent
 - Issues: https://git.falahos.my/wmj/hermes-cpanel-agent/issues
+- Wiki: https://git.falahos.my/wmj/hermes-cpanel-agent/wiki

QA-Test-Strategy.md

@@ -0,0 +1,287 @@
+# QA Test Strategy — Falah OS Ecosystem
+
+> **Status:** Active · **Owner:** @hermes-contabo · **Last Updated:** 2026-07-09
+> 
+> **Tagged Agents:** @Ody @OC @PI @hermes-contabo
+
+---
+
+## Ecosystem Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    Falah OS CE (AWS 13.140.161.244)            │
+│  ┌──────────┐  ┌──────────┐  ┌──────────────┐  ┌───────────┐  │
+│  │ Falah    │  │ iStore   │  │ App Manager  │  │ Gateway   │  │
+│  │ App:3005 │  │ :3021    │  │ :3022        │  │ :3000     │  │
+│  │ (React)  │  │(registry)│  │(docker apps) │  │           │  │
+│  └──────────┘  └──────────┘  └──────────────┘  └───────────┘  │
+└─────────────────────────────────────────────────────────────────┘
+                           │
+                    Cloudflare Tunnel
+                           │
+┌─────────────────────────────────────────────────────────────────┐
+│                 Hermes Plugin + WordPress (Ummah)               │
+│  ┌──────────────────────────────────────────────────────────┐   │
+│  │  Hermes AI Bridge Plugin                                 │   │
+│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐  │   │
+│  │  │REST API  │ │AI Prov.  │ │Scheduler │ │Nextcloud   │  │   │
+│  │  │hermes/v1 │ │(OpenClaw)│ │(WP-Cron) │ │Bridge      │  │   │
+│  │  └──────────┘ └──────────┘ └──────────┘ └────────────┘  │   │
+│  └──────────────────────────────────────────────────────────┘   │
+└─────────────────────────────────────────────────────────────────┘
+                           │
+              ┌────────────┴────────────┐
+              ▼                         ▼
+      ┌──────────────┐        ┌──────────────────┐
+      │ Ody Clone    │        │ PI Clone         │
+      │ (Content Gen)│        │ (Auto-Reply)     │
+      │ port 7000    │        │ (in WordPress)   │
+      └──────────────┘        └──────────────────┘
+              │                         │
+              └──────────┬──────────────┘
+                         ▼
+               ┌──────────────────┐
+               │ OpenClaw / OC    │
+               │ (AI Provider)    │
+               │ Mac Mini :4000   │
+               └──────────────────┘
+```
+
+## Components Under Test
+
+| Component | Agent | Location | Type | Priority |
+|-----------|-------|----------|------|----------|
+| Hermes WordPress Plugin | @hermes-contabo | cPanel / WordPress | PHP Plugin | P0 |
+| Ody Clone (Content Gen) | @Ody | Mac Mini :7000 | MCP Server | P0 |
+| PI Clone (Auto-Reply) | @PI | WordPress | Agent | P1 |
+| OpenClaw Provider | @OC | Mac Mini :4000 | MCP Server | P0 |
+| Falah OS CE Frontend | @hermes-contabo | AWS 13.140.161.244 | React + Express | P1 |
+| iStore App Registry | @hermes-contabo | Docker :3021 | Express Service | P1 |
+| App Manager | @hermes-contabo | Docker :3022 | Express Service | P1 |
+| Gitea Actions CI/CD | @hermes-contabo | Contabo | Pipeline | P0 |
+
+---
+
+## Tier 1: Hermes Plugin (Core Runtime)
+
+**Owner: @Ody @OC**
+
+| # | Test | Method | Pass Criteria | Priority |
+|---|------|--------|--------------|----------|
+| 1.1 | Plugin activation | Activate → check DB tables, options, cron | `hermes_log` table exists, `hermes_api_key` set | 🔴 P0 |
+| 1.2 | REST API auth | Call with wrong/missing/valid `X-API-Key` | 401 wrong → 401 missing → 200 valid | 🔴 P0 |
+| 1.3 | All 12 REST routes | Health, CRUD posts/users/settings/schedules/logs | Each returns correct shape | 🔴 P0 |
+| 1.4 | Content generation | `POST /content/generate` with prompt | Post created with AI content | 🔴 P0 |
+| 1.5 | AI provider switching | Switch OpenClaw ↔ OpenAI ↔ Anthropic | API calls route to correct provider | 🟡 P1 |
+| 1.6 | WP-Cron scheduling | Schedule future content → tick → verify | Post published at correct time | 🔴 P0 |
+| 1.7 | Security: no banned funcs | `exec/shell_exec/system/eval` grep | Zero hits | 🔴 P0 |
+| 1.8 | Admin settings UI | Page renders, save works, key regen works | All fields persist, new key invalidates old | 🟡 P1 |
+| 1.9 | Logging | All actions logged in `hermes_log` | Entries present with correct levels | 🟡 P1 |
+
+### API Contract Verification
+
+```
+hermes/v1:
+  ├── GET    /health              → {"status":"ok","version":"1.0.0",...}
+  ├── POST   /content/generate    → {"success":true,"post_id":N,"edit_url":"..."}
+  ├── POST   /content/schedule    → {"success":true,"task_id":N,"scheduled_date":"..."}
+  ├── GET    /posts               → [{id,title,status,date,modified,permalink}]
+  ├── GET    /posts/:id           → {id,title,content,excerpt,status,date,...}
+  ├── PUT    /posts/:id           → {"success":true,"post_id":N}
+  ├── DELETE /posts/:id           → {"success":true,"deleted":N}
+  ├── GET    /users               → [{ID,login,email,display_name,roles}]
+  ├── GET    /users/:id           → {id,login,email,display_name,roles}
+  ├── GET    /settings            → {site_name,ai_provider,ai_model,...}
+  ├── PUT    /settings            → {"success":true}
+  ├── GET    /schedules           → [{id,type,scheduled_date,status,created}]
+  └── GET    /logs?level=&limit=  → [{id,level,message,context,created_at}]
+```
+
+---
+
+## Tier 2: Ody Clone (Content Generation Agent)
+
+**Owner: @Ody @OC**
+
+| # | Test | Method | Pass Criteria | Priority |
+|---|------|--------|--------------|----------|
+| 2.1 | MCP connectivity | JSON-RPC `initialize` on `mac-mini.local:7000` | Returns `serverInfo` + capabilities | 🔴 P0 |
+| 2.2 | Tool discovery | `tools/list` after init | Exposes content-gen tools | 🔴 P0 |
+| 2.3 | Content generation | Call MCP `content_generate` tool | Returns markdown content | 🔴 P0 |
+| 2.4 | Content quality | Generate 10 Islamic finance posts | No haram content, correct terminology | 🟡 P1 |
+| 2.5 | Error handling | Missing API key, rate limits, timeouts | Structured error responses | 🔴 P0 |
+| 2.6 | Prompt injection | Malicious prompt payloads | Prompt boundaries respected, no system prompt leak | 🔴 P0 |
+
+### MCP Tool Contract
+
+```
+tools:
+  content_generate:
+    input:  {prompt: string, tone?: string, length?: number}
+    output: {content: string, title?: string, word_count: number}
+  
+  content_schedule:
+    input:  {prompt: string, schedule_date: string, post_type?: string}
+    output: {task_id: number, scheduled_date: string}
+  
+  health_check:
+    input:  {}
+    output: {status: string, uptime: number, model: string}
+```
+
+---
+
+## Tier 3: PI Clone (Auto-Reply Agent)
+
+**Owner: @PI**
+
+| # | Test | Method | Pass Criteria | Priority |
+|---|------|--------|--------------|----------|
+| 3.1 | Comment auto-reply | Create comment on WordPress post | PI replies within cron window | 🟡 P1 |
+| 3.2 | Reply relevance | Test comments about Islamic finance | Reply is contextually appropriate | 🟡 P1 |
+| 3.3 | Spam guard | Spam comment → PI should not reply | No reply generated for spam | 🟡 P1 |
+| 3.4 | Rate limiting | 50 comments in 1 min → PI behavior | Does not reply to all, respects limits | 🟢 P2 |
+
+---
+
+## Tier 4: OpenClaw Provider (AI Inference)
+
+**Owner: @OC**
+
+| # | Test | Method | Pass Criteria | Priority |
+|---|------|--------|--------------|----------|
+| 4.1 | MCP connectivity | JSON-RPC `initialize` on `mac-mini.local:4000` | Returns capabilities | 🔴 P0 |
+| 4.2 | Model availability | `tools/list` | Provider models listed | 🔴 P0 |
+| 4.3 | Content generation | Generate via OpenClaw API | Returns text completion | 🔴 P0 |
+| 4.4 | Model switching | Switch model in Hermes admin → verify | Different model used | 🟡 P1 |
+
+---
+
+## Tier 5: Falah OS CE + Docker Services
+
+**Owner: @hermes-contabo**
+
+| # | Test | Method | Pass Criteria | Priority |
+|---|------|--------|--------------|----------|
+| 5.1 | Docker build | `docker compose build` | Exit 0, no errors | 🔴 P0 |
+| 5.2 | All services healthy | `docker compose up -d` + `/health` on each | All 3 services healthy | 🔴 P0 |
+| 5.3 | Falah App UI | Load React app in browser | All modules visible (Zakat, Waqf, Qurban, Sadaqah, Infaq) | 🟡 P1 |
+| 5.4 | iStore + Gitea connection | List available apps via iStore | Returns app list | 🟡 P1 |
+| 5.5 | App Manager | Install → verify container → uninstall | Clean lifecycle | 🟡 P1 |
+| 5.6 | Window manager | Drag, dock, context menu, search bar | All interactions work | 🟢 P2 |
+| 5.7 | Casdoor auth | OIDC login → redirect → callback | Authenticated session | 🔴 P0 |
+
+---
+
+## Tier 6: CI/CD Pipeline (Gitea Actions)
+
+**Owner: @hermes-contabo**
+
+| # | Test | Workflow | Pass Criteria | Priority |
+|---|------|----------|--------------|----------|
+| 6.1 | PHP lint | `ci.yml` | All `.php` files parse clean | 🔴 P0 |
+| 6.2 | Banned function check | `ci.yml` | No `exec/shell_exec/system/eval` hits | 🔴 P0 |
+| 6.3 | Secret scan | `ci.yml` | No hardcoded passwords/tokens | 🔴 P0 |
+| 6.4 | File permissions check | `ci.yml` | No world-writable files outside `.git` | 🟡 P1 |
+| 6.5 | Schema validation | `ci.yml` | `hermes/v1` namespace declared, all classes in autoloader | 🔴 P0 |
+| 6.6 | cPanel deploy | `deploy.yml` | Git push to cPanel succeeds | 🔴 P0 |
+| 6.7 | WP API fallback | `deploy.yml` | Plugin zip upload works when cPanel fails | 🟡 P1 |
+| 6.8 | Post-deploy verify | `deploy.yml` | Namespace + plugin presence on target | 🔴 P0 |
+| 6.9 | Health check | `health.yml` (every 30m) | REST endpoint responds | 🟡 P1 |
+| 6.10 | Sync | `sync.yml` (every 6h) | cPanel sync + Nextcloud artifacts | 🟢 P2 |
+
+---
+
+## Tier 7: End-to-End Integration
+
+**Owner: @Ody @OC @PI @hermes-contabo**
+
+| # | Scenario | Steps | Expected | Priority |
+|---|----------|-------|----------|----------|
+| 7.1 | **Full content pipeline** | Dev push → CI → Deploy → WP → Hermes → Ody → OC → WP post | Post published with AI content | 🔴 P0 |
+| 7.2 | **Scheduled content** | Schedule via API → WP-Cron tick → Ody → OC → Post | Post created at correct time | 🔴 P0 |
+| 7.3 | **Nextcloud sync** | Upload file → Hermes WebDAV → Download | Round-trips successfully | 🟡 P1 |
+| 7.4 | **Cloudflare WAF** | Call Hermes API through CF tunnel | Request passes, `ody.falahos.my` reachable | 🔴 P0 |
+| 7.5 | **Secret rotation** | Regenerate API key → old calls fail → new calls work | Zero downtime on rotation | 🔴 P0 |
+| 7.6 | **Container lifecycle** | App Manager install → run → uninstall → verify | No orphaned containers | 🟡 P1 |
+| 7.7 | **Load test** | 50 concurrent content generation requests | No crash, graceful degradation | 🟡 P1 |
+
+---
+
+## 🚀 Pre-Deploy Smoke Suite (< 5 min)
+
+Run before every production push:
+
+```bash
+# 1. Docker stack
+docker compose build && docker compose up -d
+for port in 3005 3021 3022; do
+  curl -sf "http://localhost:$port/health" || echo "FAIL: port $port"
+done
+
+# 2. Hermes API health
+curl -sf -H "X-API-Key: $HERMES_KEY" \
+  "https://ummah.falahos.my/wp-json/hermes/v1/health" \
+  | grep '"status":"ok"' || echo "FAIL: Hermes health"
+
+# 3. Content generation
+curl -sf -X POST -H "X-API-Key: $HERMES_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"prompt":"Test: Write 3 sentences about zakat"}' \
+  "https://ummah.falahos.my/wp-json/hermes/v1/content/generate" \
+  | grep '"success":true' || echo "FAIL: Content gen"
+
+# 4. MCP - Ody
+echo '{"jsonrpc":"2.0","id":1,"method":"initialize"}' | \
+  websocat ws://mac-mini.local:7000/mcp || echo "FAIL: Ody MCP"
+
+# 5. MCP - OC
+echo '{"jsonrpc":"2.0","id":1,"method":"initialize"}' | \
+  websocat ws://mac-mini.local:4000/mcp || echo "FAIL: OC MCP"
+```
+
+---
+
+## 📊 Test Result Tracking
+
+| Suite | Tests | Pass | Fail | Blocked | Coverage |
+|-------|-------|------|------|---------|----------|
+| Plugin Core | 9 | — | — | — | — |
+| Ody Agent | 6 | — | — | — | — |
+| PI Agent | 4 | — | — | — | — |
+| OC Provider | 4 | — | — | — | — |
+| Docker/Frontend | 7 | — | — | — | — |
+| CI/CD Pipeline | 10 | — | — | — | — |
+| E2E Integration | 7 | — | — | — | — |
+| **Total** | **47** | — | — | — | — |
+
+---
+
+## ⚠️ Known Blockers (Pre-QA)
+
+| # | Issue | Owner | Status |
+|---|-------|-------|--------|
+| B1 | Cloudflare WAF blocks Contabo from WP REST API | @hermes-contabo | Open |
+| B2 | WordPress Application Password needed for deploy | @Ody | Open |
+| B3 | Openclaw MCP server not running (port 4000 closed) | @OC | Open |
+| B4 | Odysseus MCP server not running (port 7000 closed) | @Ody | Open |
+
+---
+
+## 🛠 Testing Tools
+
+| Tool | Used For |
+|------|----------|
+| PHPUnit + WP_Mock | PHP unit tests |
+| Postman / Bruno | REST API collection + automation |
+| MCP Inspector | Ody & PI MCP tool testing |
+| `docker compose ps + /health` | Service health verification |
+| Playwright | React UI + window manager tests |
+| Gitea Actions logs | CI/CD failure inspection |
+| k6 | Load testing agent endpoints |
+| trufflehog | Secret leak detection |
+
+---
+
+*Page auto-generated by @hermes-contabo · Edit at `https://git.falahos.my/wmj/hermes-cpanel-agent/wiki/QA-Test-Strategy`*