Document AI — from PDF claim LLM extraction to rule-based decisions and audit trails

Workshop goal

By the time you finish this Workshop, you will have walked through one full cycle of the following flow inside D.Hub.

• Load a single scenario into portal so that two sub-collections (inbox and processed), 3 datasets, 3 code nodes, 3 pipelines, an ontology with 3 entities and 2 relations, a dashboard, and one knowledge are all registered together.
• Inspect the metadata-only claim_documents (object type) dataset, which holds PDF claim form metadata.
• Use the claim_extraction pipeline to run OCR + LLM extraction on one PDF at a time, surfacing structured fields like incident date, claim amount, diagnosis code, policy number, with a per-field confidence_score.
• Use the decision_workflow pipeline's 6-rule engine to assign one of APPROVED / REJECTED / ESCALATED, and leave a why trail in decision_log.
• Expand a Document / Claimant / Policy graph to surface a single claimant's claim cluster.
• Read the claims_processing dashboard for throughput, decision distribution, and SLA compliance in one view.

The Workshop centers on the coupling pattern of LLM-driven automatic extraction with rule-based decisioning. Recommended duration: 90 minutes.

Prerequisites

• An analyst or engineer account with access to D.Hub portal (Editor or higher)
• ~42 KB of download room for one scenario zip

No terminal, Python, or dhub2-examples clone needed.

1. Load the scenario (10 min)

Go to Collections → more (⋯) menu → Import (가져오기) and upload the zip.

Following manifest.json, the import creates the following in order:

1. Two collections — inbox (alias: Incoming Claim Documents), processed (alias: Processed Claims)
2. Three datasets — claim_documents (inbox, object type), extracted_fields (processed), decision_log (processed)
3. Three codes (Python) — document_parser, claim_adjudicator, build_docai_ontology
4. Three pipelines — claim_extraction, decision_workflow, ontology_materialization
5. Ontology — three entities (Document, Claimant, Policy), two relations (filed_by, covered_by)
6. One knowledge — claims_handbook (claim processing guidelines)
7. One dashboard — claims_processing

Step 1 is complete once inbox and processed show in the left tree. The received → processed time flow is intentionally expressed at the collection layer.

2. Browse the claim documents (10 min)

In the inbox collection, claim_documents is an object-type dataset. Each row corresponds to one PDF file; the columns are metadata, not the PDF body itself.

• claim_id — Unique claim identifier
• document_key — PDF key in the S3 / object store
• claimant_name — Claimant's name
• policy_number — Insurance policy number
• received_at — Timestamp
• page_count — Page count
• source_channel — email / web_portal / mobile_app

The PDF body itself is fetched and processed by document_parser in the next step via the document key. This separation — metadata dataset + object store — is the standard pattern for large-volume unstructured data.

In the processed collection, extracted_fields and decision_log start empty (0 rows). The next-step pipelines fill the first rows.

3. LLM-driven field extraction (20 min)

In the processed collection's Pipelines section, open claim_extraction. One node — document_parser reads claim_documents row by row and processes one PDF at a time.

The document_parser script's two-stage flow:

1. OCR (pdfplumber) — Extract the text layout from PDF pages. Falls back to OCR for scanned images.
2. LLM extraction — Send the extracted text to the LLM with a structured-extraction prompt and receive the following fields as JSON:
   • incident_date — Incident date
   • claim_amount — Claim amount
   • diagnosis_code — ICD diagnosis code (for medical claims)
   • policy_number — Policy number
   • provider_name — Provider name (when applicable)
   • confidence_score — 0–1, the LLM's self-assessed confidence

The key column is confidence_score. The prompt is designed to elicit honest uncertainty, and the next-step decision engine auto-escalates any claim with confidence below 0.6.

Press Run. When it finishes, open the Preview tab on extracted_fields. The row count should match claim_documents, and confidence_score should distribute in the 0–1 range.

4. 6-rule decision engine (15 min)

Open the decision_workflow pipeline. One node — claim_adjudicator reads extracted_fields and produces decision_log.

The claim_adjudicator evaluates 6 rules in priority order.

The key design choice is defaulting to ESCALATED. Auto-approval only accepts high-confidence, low-amount claims; everything else goes to humans. The pattern reflects the false-positive cost ≫ false-negative cost asymmetry of insurance.

Press Run. When it finishes, open decision_log. On the seed data, decisions distribute roughly APPROVED : REJECTED : ESCALATED ≈ 4 : 2 : 4. The reason column carries a one-line explanation per row — the core column for audit and regulatory response.

5. Ontology + graph exploration (10 min)

Run ontology_materialization once. build_docai_ontology reads three datasets in batch mode and lands the following five artifacts in upsert mode.

In the Graph explorer, surface a single claimant's claim cluster.

MATCH (c:Claimant)<-[:filed_by]-(d:Document)-[:covered_by]->(p:Policy)
WHERE c.claim_count >= 3
RETURN c, d, p
LIMIT 25

A claimant with three or more claims appears, with all their documents and the policies they cover. When several Document nodes share one Policy, that's the first visual signal of repeat-claim patterns.

6. Dashboard + audit trail (15 min)

Open claims_processing (alias: Claims processing dashboard). Eight widgets share the screen.

• Claims Received (statistic) — COUNT(*) over claim_documents
• Decisions Made (statistic) — COUNT(*) over decision_log
• Avg Extraction Confidence (statistic) — AVG(confidence) on extracted_fields (to 2 decimal places)
• SLA Met Rate (statistic) — % rate of sla_met = true on decision_log
• Decision Distribution (donut) — Counts per decision on decision_log (APPROVED / REJECTED / ESCALATED distribution)
• Escalations by Reason (bar) — Counts per reason on decision_log, descending
• Claims by Channel (bar) — Counts per source_channel on claim_documents (email / web_portal / mobile_app)
• Recent Claims (data table) — 100 most recent rows by received_at desc

The operational signals are two statistics + two distributions. If Avg Extraction Confidence drops below 0.7, audit the LLM prompt or OCR quality; if SLA Met Rate is low, the §4 decision engine is routing too many cases to ESCALATED. The first bar of Escalations by Reason tells you in one line which rule fires most.

7. Next steps and a retrospective (10 min)

By this point you've walked through five flows on a single scenario.

• Metadata-dataset + object-store separation (object type pattern)
• OCR + LLM structured extraction with confidence_score signaling
• Decision engine's ESCALATED-as-default asymmetry
• Graph view of repeat-claim clusters
• Audit trail via reason column

Pick the most appealing direction:

• The refund-approval Workshop covers the HITL agent variant of the same shape — default ESCALATED → human review → automated execution maps cleanly to insurance.
• The Analyst Path Managing widgets lesson — rebuild the Escalations by Reason widget's SQL by hand.
• Map the document → structure → decision shape to your own domain (contract review, invoice processing, résumé screening).

Verification checklist

• Both inbox and processed collections appear in the collection tree, with claim_documents set as object type.
• After claim_extraction, the confidence_score column on extracted_fields is in the 0–1 range, with at least one or two rows in 0.4–0.55.
• After decision_workflow, the decision_log decisions are distributed across APPROVED / REJECTED / ESCALATED and each row has a populated reason text.
• In the Graph explorer, starting from one Claimant you can expand the 3-hop path filed_by → Document → covered_by → Policy.
• The claims_processing dashboard's Decision Distribution widget shows ESCALATED as the largest slice (a direct consequence of the default ESCALATED design).