Protocol Compiler
A production-grade system that decompiles clinical trial protocol PDFs into the structured data needed to build E-Source systems, CDISC-compliant study configurations, eSource forms, and annotated CRF exports. It combines AI-driven extraction with deterministic validation, rule-based mapping, and human-review gating.
The Problem
Building an electronic data-capture system for a clinical study starts with a protocol PDF — often hundreds of pages containing visits, procedures, eligibility criteria, and complex edit checks. Manually translating that document into study-build artifacts is slow, error-prone, and inconsistent:
- Visits, forms, and fields are buried in narrative text and tables.
- Numeric ranges, temporal windows, and allowed values vary by study and are easy to miss.
- CDISC/CDASH conventions must be applied consistently across dozens of domains.
- Low-confidence AI extractions can silently introduce fabricated entities.
- Collaborators need shared workspaces with clear roles, not isolated owner-only access.
Solution Overview
PDF Ingestion & Parsing
- Magic-bytes validation rejects non-PDFs, corrupted files, and password-protected uploads
- PyMuPDF + pdfplumber extract plain text, tables, and visit-schedule matrices
- Page classification (text / image / mixed) with optional OCR fallback for scanned protocols
- Raw text and tables persisted for transparent debugging and provenance
AI-Driven Extraction
- Multi-pass focused prompting: visits, forms, item groups, fields, code lists, edit checks
- Provider-agnostic LLM backend: OpenAI, Groq, Gemini, Ollama, or mock for testing
- Per-entity confidence scores with source grounding (span, chunk, page reference)
- Centralized rate limiting, daily quotas, retries, and provider failover
- Deterministic response cache for quota-free reruns
Generalized Constraint Extraction
- Deterministic scan for numeric ranges, thresholds, and temporal windows
- Allowed categorical values and sample/storage/dosing constraints
- Constraints linked to matching entities; unmatched ones surfaced for review
- Not hardcoded to vital signs or labs — works across any item or section
Reference Enrichment & Mapping
- Static CDISC/CDASH library for canonical domains and SAS field names
- Dynamic Annotated CRF library learned from uploaded reference PDFs
- Versioned LearnedTemplates with provenance tracking and rollback
- Priority resolution when static templates, CRFs, and LLM output conflict
Validation, Review & Export
- Rule-based validation engine with ERROR / WARNING / INFO findings
- Export gated by validation status, unresolved review, and confidence thresholds
- Multi-format export: JSON, CSV, Excel, ODM XML, Annotated CRF HTML
- Immutable study revisions with snapshot creation, comparison, and revert
Collaboration & Access Control
- Study-level collaborator roles: viewer, reviewer, editor, admin, owner
- Centralized permission enforcement via reusable access-control helpers
- 404-not-403 pattern prevents leaking existence of unshared studies
- Full audit trail of uploads, extractions, reviews, and exports
Architecture
The backend is an async FastAPI application with SQLAlchemy 2.0 + Alembic on PostgreSQL, orchestrated by Celery for long-running extraction jobs. The React + TypeScript frontend uses Vite, TailwindCSS, and Zustand, with shared UI primitives across every view.
- Backend: FastAPI, SQLAlchemy 2.0, Alembic, async psycopg, Pydantic 2, pytest
- Frontend: React 18, TypeScript 5.4, Vite 5.3, TailwindCSS 3.4, Zustand, React Router 6
- AI/LLM: OpenAI / Groq / Gemini / Ollama with unified rate limiter and fallback chain
- PDF Parsing: PyMuPDF + pdfplumber, optional Azure Document Intelligence
- Export: openpyxl (Excel), stdlib csv/json/xml
- Testing: 1,200+ backend tests, Vitest + Testing Library frontend tests
Tech Stack
Python 3.12
FastAPI
SQLAlchemy 2.0
Alembic
PostgreSQL
Celery
Redis
React 18
TypeScript 5.4
Vite 5.3
TailwindCSS 3.4
Zustand
OpenAI / Groq / Gemini / Ollama
PyMuPDF
pdfplumber
Screenshots
Challenges & Solutions
- Hallucination in LLM outputs: Critical entities receive confidence scores and source-grounding checks. Anything fabricated or low-confidence is routed to human review before export, and export is blocked until resolved.
- Variable provider quotas and failures: A centralized rate limiter with per-provider RPM buckets, daily caps, and retry logic ensures one exhausted provider does not kill a run; failover chains fall back through remote and local models.
- Study-specific conventions: Static CDISC templates alone cannot capture sponsor patterns. Annotated CRF upload and LearnedTemplate versioning let the system absorb approved study designs and reuse them safely.
- Complex PDF layouts: Multi-column pages, headers/footers, and scanned image pages required layered parsing: reading-order heuristics, table extraction, and optional OCR, all feeding the same downstream extractor.
- Shared workspace security: Centralized access-control helpers enforce study-level roles, while 404 responses for unauthorized requests prevent information leakage.
Outcomes & Impact
- Turns protocol PDF review from days of manual reading into a structured first draft in minutes.
- Reduces missed ranges, thresholds, and eligibility criteria through deterministic constraint extraction.
- Enforces consistent CDISC/CDASH conventions across exported study designs.
- Provides an auditable, versioned path from raw protocol → reviewed design → export artifact.
- Supports safe self-learning: approved corrections become reusable templates without retraining models.
Lessons Learned
- Multi-pass, focused prompts are far more reliable than asking a single LLM call to produce an entire study design.
- Every AI-extracted entity needs source evidence; without it, reviewers cannot trust or efficiently verify the output.
- Deterministic constraint extraction complements LLM extraction by catching numeric patterns the model may paraphrase incorrectly.
- Export gating and revision snapshots are essential in regulated workflows where artifacts must be defensible.
Future Improvements
- Diff viewer comparing any two study revisions side-by-side.
- Study-template cloning to bootstrap new protocols from similar historical designs.
- Live collaboration cursors and review comments in the study workspace.
- Regulatory-submission-ready annotated CRF generation with sponsor branding.