Protocol Decompiler
A clinical protocol intelligence platform that transforms complex trial protocol PDFs into structured, traceable, and reviewable study-build artifacts. It reconstructs Schedule of Assessments tables, extracts visits, procedures, forms, fields, eligibility criteria, laboratories, dosing events, and constraints, then presents a ClinSpark-oriented blueprint with evidence, confidence, diagnostics, validation, and export controls.
The Problem
Building an electronic data-capture or eSource system starts with a protocol PDF that was written for human interpretation, while the study system requires exact relationships between visits, timepoints, procedures, forms, fields, labs, dosing events, eligibility criteria, and edit checks. Manual translation is slow and fragile:
- Schedule tables can span pages, mix periods, and include footnotes that change the meaning of a visit or procedure.
- PDF text order often differs from visual order, especially in dense Schedule of Assessments tables.
- Eligibility criteria, laboratory panels, ranges, code lists, and dependencies must remain traceable to source pages.
- AI extraction is useful, but unsupported assumptions and low-confidence entities must be visible and reviewable.
- Builders need implementation-oriented blueprints, not just summaries of protocol text.
Solution Overview
Protocol Ingestion
- Validates uploads, streams large PDFs to storage, calculates checksums, and rejects invalid or duplicate inputs
- Classifies pages and prepares cleaned AI payloads while preserving original page text for auditability
- Identifies schedule, visit, procedure, eligibility, treatment, safety, laboratory, PK, and specimen sections using semantic and structural signals
Schedule Reconstruction
- Rebuilds Schedule of Assessments matrices instead of flattening them into paragraphs
- Preserves periods, cohorts, visits, intra-visit timepoints, procedures, notes, and continuation-page relationships
- Prevents common mistakes such as merging Screening with Admission/Day -1 or collapsing timed Day 1 procedures
Evidence-First Model
- Builds deterministic study entities for metadata, visits, procedures, forms, fields, criteria, interventions, labs, ranges, code lists, and edit checks
- Stores source wording, page references, confidence, diagnostics, and build gaps for important results
- Uses explicit unresolved states instead of fabricating missing values
AI-Assisted Extraction
- Runs focused passes for visits, procedures, forms, item groups, fields, code lists, criteria, and edit checks
- Supports OpenRouter, OpenAI, Groq, Gemini, Ollama, and mock-provider paths
- Uses structured output handling, rate limiting, retries, caching, fallback routing, and confidence scoring
ClinSpark Blueprint
- Presents Study to Period/Visit to Timepoint to Procedure to Form to Field relationships
- Highlights laboratory panels, eligibility criteria, evidence, readiness, unresolved fields, and build gaps
- Generates builder-oriented packages for ClinSpark, EDC, eSource, and CDISC-oriented workflows
Validation & Export
- Validates referential integrity, semantic consistency, evidence coverage, duplicates, orphaned records, and export readiness
- Gates unsafe exports until review-required or unresolved artifacts are corrected
- Exports JSON, CSV, XLSX, ODM-XML, annotated CRF HTML, raw text, and ClinSpark-oriented packages
Architecture
The backend is a Python/FastAPI application with SQLAlchemy persistence, background job processing, structured logging, authentication, authorization, storage controls, and REST APIs. The frontend is a React, TypeScript, and Vite application with study navigation, upload status, review panels, blueprint views, diagnostics, exports, and accessible UI components.
- Backend: FastAPI, SQLAlchemy, Pydantic, background jobs, JWT auth, structured logs
- Frontend: React, TypeScript, Vite, TailwindCSS, review and blueprint interfaces
- AI/LLM: OpenRouter, OpenAI, Groq, Gemini, Ollama, and mock-provider paths
- PDF Parsing: Layout-aware text/table parsing with OCR fallback concepts
- Reference Data: ODM-XML, annotated CRF, CDISC/CDASH-oriented enrichment
- Deployment: Render, Cloudflare Pages, Neon, and hosted LLM provider guidance
Tech Stack
Python
FastAPI
SQLAlchemy 2.0
Alembic
PostgreSQL
Celery
Redis
React 18
TypeScript
Vite
TailwindCSS
Zustand
LLM Orchestration
ODM-XML
CDISC
Screenshots
Challenges & Solutions
- Schedule table ambiguity: The parser treats SoA tables as matrices, preserving visual column relationships, repeated headers, footnotes, period bands, and intra-visit timepoints.
- AI uncertainty: Extracted entities retain evidence, confidence, diagnostics, and review-required states so unsupported assumptions cannot silently become build artifacts.
- Long-running processing: Asynchronous jobs use staged progress, checkpoints, cancellation, retries, re-upload handling, provider throttling, caching, and recovery controls.
- Reference standardization: ODM-XML and annotated CRF references enrich forms, fields, code lists, methods, and controlled terminology while preserving ambiguity and provenance.
- Export readiness: Validation catches missing relationships, duplicate records, orphaned objects, unresolved windows, laboratory gaps, and low-confidence artifacts before export.
Outcomes & Impact
- Creates a reviewable starting point for protocol-to-study-build work instead of an opaque PDF summary.
- Preserves source evidence, page references, confidence, and diagnostics for important extracted artifacts.
- Separates extracted, inferred, unresolved, and approved information so builders can correct gaps before configuration.
- Combines document intelligence, clinical modeling, LLM orchestration, validation, auditability, and deployment concerns in one product.
Lessons Learned
- Protocol parsing should prefer diagnosable unresolved states over confident-looking fabricated values.
- Schedule of Assessments tables need layout-aware handling; text extraction alone loses too many relationships.
- Focused extraction passes are easier to validate and repair than a single broad model response.
- Export gates are product features, not friction, when the domain requires defensible artifacts.
Future Improvements
- Add deeper support for image-based protocols and difficult scanned tables.
- Expand protocol fixture coverage across more sponsor formats.
- Add richer side-by-side review for extracted artifacts and their source evidence.
- Harden deployment and validation evidence for broader regulated use.