14 - QA Checklist (Webhooks Module)
- 1. Endpoint contract (C)
- 2. Delivery pipeline (C)
- 3. Signature verification (C)
- 4. Retry & test (H)
- 5. Payload size & robustness (M)
- 6. Schema/bookkeeping gaps (M)
- 7. Resilience (M)
- 8. Manual QA script (smoke)
Test plan for the implemented backend surface. Baseline conventions in 00-shared/10; module items below are specific to webhook delivery semantics. Each item cites its source contract. Severity: C = critical, H = high, M = medium.
1. Endpoint contract (C)
| # | Check | Expected | Source |
|---|---|---|---|
| E-1 | CRUD happy path | POST → 201 doc; GET list newest-first; GET :id; PATCH subset; DELETE → { message: 'Webhook deleted' } and doc gone from list | webhooks.controller.ts:21-50 |
| E-2 | 404s | GET/PATCH/DELETE unknown id → 404 'Webhook not found' | webhooks.service.ts:39-43, 45-48, 50-53 |
| E-3 | Permission matrix | webhook.create/read/update/delete enforced per endpoint (permissions.constants.ts:89-92) | webhooks.controller.ts:22, 28, 34, 40, 46, 53, 59, 66, 73, 79, 86 |
| E-4 | Validation | empty name/url/secret → 400; events: [] → 400 (ArrayMinSize(1)); bad url → 400; enabled non-boolean → 400 | create-webhook.dto.ts:11-34 |
| E-5 | Logs | GET :id/logs returns ≤ 50, newest first; includes payload, status, responseCode?, responseBody? | webhooks.service.ts:88-93, webhook-delivery-log.schema.ts:12-38 |
| E-6 | Metrics | counts reconcile: total = success + failed + pending for same webhook | webhooks.service.ts:149-154 |
| E-7 | Tenant isolation | tenant B cannot read/update/delete/pause tenant A webhooks (scoped filter) | base.repository.ts:20-30 |
2. Delivery pipeline (C)
| # | Check | Expected | Source |
|---|---|---|---|
| D-1 | Event fan-out | emitting event X creates one deliver job per enabled webhook subscribed to X | webhooks.service.ts:55-86, webhook.repository.ts:17-29 |
| D-2 | No fan-out | disabled / soft-deleted / unsubscribed webhooks receive nothing | webhook.repository.ts:23-28 |
| D-3 | Timeout | endpoint that sleeps > 10 s → attempt failed (AbortSignal.timeout(10000)) | webhook-delivery.worker.ts:81 |
| D-4 | Retry counts | failing endpoint observed 3 job attempts with exponential backoff (5 s, 10 s) | webhooks.service.ts:80-83 |
| D-5 | Status recording | 2xx → success with responseCode+body; non-2xx → failed; network error → failed | webhook-delivery.worker.ts:84-99 |
| D-6 | Payload fidelity | receiver gets exactly JSON.stringify(payload) of the emitted event payload | webhook-delivery.worker.ts:70, 80, domain-event.interface.ts:7 |
| D-7 | Queue presence | queue named webhook-deliver registered; worker attached | queue.constants.ts:14, webhooks.module.ts:22, webhook-delivery.worker.ts:9 |
3. Signature verification (C)
| # | Check | Expected | Source |
|---|---|---|---|
| S-1 | Signature correctness | receiver computes HMAC-SHA256(rawBody, secret) hex and matches X-Webhook-Signature | webhook-delivery.worker.ts:77, 103-105 |
| S-2 | Header presence | every POST has Content-Type: application/json, X-Webhook-Signature, X-Webhook-Event | webhook-delivery.worker.ts:76-79 |
| S-3 | Secret mismatch | wrong stored secret → signature mismatch (receiver rejects) - expected, verifies scheme | webhook.schema.ts:19-20 |
| S-4 | Secret rotation | PATCH secret → subsequent deliveries signed with new secret | update-webhook.dto.ts:4, webhooks.service.ts:45-48 |
| S-5 | Known gap (H) | no X-Webhook-Timestamp / replay window - document; verification must not assume one | webhook-delivery.worker.ts:103-105 |
4. Retry & test (H)
| # | Check | Expected | Source |
|---|---|---|---|
| R-1 | Retry no-failure | POST :id/retry with zero failed logs → 404 'No failed deliveries to retry' | webhooks.service.ts:103-108 |
| R-2 | Retry payload | retry re-sends the latest failed attempt's eventType + payload verbatim | webhooks.service.ts:110-118 |
| R-3 | Retry correlation | retried job carries correlationId: '' - receiver must tolerate empty string | webhooks.service.ts:117 |
| R-4 | Test event | POST :id/test → attempt with eventType: 'WebhookTested', payload: { test: true, webhookId } | webhooks.service.ts:130-138 |
| R-5 | Async replies | both endpoints reply { message: '… queued' } before any delivery happens | webhooks.controller.ts:62, 69 |
| R-6 | Double-record bug (H) | on non-2xx the worker records failed twice; second call drops responseCode/responseBody - verify final log has code+body or flag as bug | webhook-delivery.worker.ts:84-99 |
5. Payload size & robustness (M)
| # | Check | Expected | Source |
|---|---|---|---|
| P-1 | Large payload | e.g. 1 MB homework/results payload delivered intact (no truncation) | webhook-delivery.worker.ts:70-82 |
| P-2 | Non-JSON-safe payload | payload containing strings/numbers only (Record<string, unknown>); verify serialization edge cases | webhook-delivery-log.schema.ts:15-16 |
| P-3 | Response body capture | 5xx with HTML body - stored raw, client must render escaped | webhook-delivery.worker.ts:88 |
| P-4 | URL edge cases | require_tld: false allows http://localhost:3000/hook and IPs - keep receiver in same tenant network (SSRF exposure is a security review item, M) | create-webhook.dto.ts:17-18 |
6. Schema/bookkeeping gaps (M)
| # | Check | Expected | Source |
|---|---|---|---|
| G-1 | attemptCount | currently always 0 - assert current behavior, track as bug when worker increments | webhook-delivery-log.schema.ts:31-32 |
| G-2 | lastTriggeredAt / failureCount | never written - surfaces can't sort by activity; track | webhook.schema.ts:25-29 |
| G-3 | Log-per-attempt | retries create new log rows (not updates) - verify UI handles duplicate-ish rows | webhooks.service.ts:62-67, 84-89 |
| G-4 | Free-form events | typo in events silently never fires (exact match, webhook.repository.ts:27) - client picker mitigates; no server whitelist | create-webhook.dto.ts:20-24 |
| G-5 | Secret exposure | plaintext secret in every CRUD response - flag to reviewers; UI must not log it | webhooks.service.ts:31-47 |
7. Resilience (M)
| # | Check | Expected | Source |
|---|---|---|---|
| Q-1 | Redis down at enqueue | queue.add failure surfaces as 500 - webhook creation still persists (fan-out is post-create) | webhooks.service.ts:69-84 |
| Q-2 | Redis down at delivery | jobs stay in BullMQ; worker reconnect replays - verify no duplicate double-record corruption | webhooks.module.ts:22 |
| Q-3 | Idempotency | duplicate event emission produces duplicate deliveries (no dedup) - document for receivers | webhooks.service.ts:55-86 |
| Q-4 | Metrics under load | counts via 4 countDocuments - fine at tenant scale; revisit with pagination (planned) | webhooks.service.ts:149-154 |
8. Manual QA script (smoke)
- Create webhook with 2 events + test receiver (echo endpoint).
- Trigger both events → expect 2 deliveries,
X-Webhook-Eventmatching. - Test ping →
WebhookTestedlog row within seconds. - Point URL at a 500 endpoint → watch 3 attempts →
failed; metrics reconcile. POST :id/retry→ verify original payload resent.- Pause → trigger event → no delivery. Resume → trigger → delivery.
- Delete → absent from list; logs endpoint 404 (webhook deleted).
- Cross-tenant: second tenant cannot see or act on the webhook.