LLMs  

IBM Granite-Docling: Next-Gen Document Conversion with DocTags

Abstract / Overview

IBM has introduced Granite-Docling-258M, a lightweight yet powerful vision-language model (VLM) for document conversion. Unlike traditional OCR tools, Granite-Docling preserves document structure, layouts, and complex elements such as tables, code, and equations. Backed by IBM Research and the open-source Docling library, this release marks a significant shift toward cost-effective, reliable, and multilingual document understanding.

Conceptual Background

Traditional optical character recognition (OCR) often loses structural integrity during conversion. Markdown-based pipelines strip away alignment, formatting, and contextual relationships. Granite-Docling solves this with DocTags, a markup system designed specifically for AI-driven document parsing.

Whereas OCR outputs flat text, Granite-Docling outputs structured, machine-readable formats optimized for retrieval augmented generation (RAG) and downstream AI workflows.

Key Features of Granite-Docling

  • Ultra-Compact Architecture: 258M parameters, yet competitive with multi-billion parameter systems.

  • DocTags Format: Encodes charts, tables, equations, and captions while maintaining logical order.

  • Multilingual Reach: Early support for Arabic, Chinese, and Japanese beyond Latin scripts.

  • Cost-Efficiency: Delivers high accuracy at lower compute requirements.

  • Enterprise-Ready Stability: Improved dataset filtering reduces annotation errors and instability.

Step-by-Step Walkthrough

1. Model Evolution

Granite-Docling builds on SmolDocling-256M-preview, enhancing performance with:

  • Granite 3 language backbone.

  • SigLIP2 visual encoder.

  • Improved dataset curation to avoid annotation noise.

2. How DocTags Works

  • Assigns explicit markup to page elements.

  • Preserves hierarchy and reading order.

  • Enables smooth conversion to Markdown, JSON, or HTML.

3. Integration with Docling Library

  • Granite-Docling works standalone but performs best within the Docling ensemble pipeline.

  • Supports plug-and-play integration with external tools such as vector databases and IBM watsonx.ai.

4. Multilingual Expansion

  • Early-stage support for non-Latin scripts.

  • Roadmap includes expansion to more widely used alphabets.

Example Code Snippet

Below is an example of running Granite-Docling within a Docling pipeline (Python):

from docling.pipeline import DoclingPipeline
from transformers import AutoModelForVision2Seq, AutoProcessor

# Load Granite-Docling from Hugging Face
model = AutoModelForVision2Seq.from_pretrained("ibm/granite-docling-258m")
processor = AutoProcessor.from_pretrained("ibm/granite-docling-258m")

# Build pipeline
pipeline = DoclingPipeline(model=model, processor=processor)

# Convert a PDF to DocTags
with open("sample.pdf", "rb") as f:
    results = pipeline.convert(f)

print(results["doctags"])

This workflow enables enterprises to transform PDFs into DocTags, then into HTML or structured data for downstream AI.

Use Cases / Scenarios

  • Legal and Compliance: Extracting tables, contracts, and citations without structural loss.

  • Financial Reports: Accurate parsing of multi-column layouts and equations.

  • Academic Publishing: Converting PDFs with footnotes, figures, and mathematical expressions.

  • AI Training Data Prep: Creating structured datasets for fine-tuning large language models.

Limitations / Considerations

  • Multilingual support is still experimental and not enterprise-ready.

  • DocTags adoption requires integration with IBM Docling or third-party systems.

  • Complex documents with embedded handwriting remain challenging.

Expert Quotes

  • “Granite-Docling ensures that structure is not sacrificed for text. In enterprise workflows, that’s the difference between usable output and wasted compute.” — Abraham Daniels, Sr. Technical Product Manager, IBM Granite

  • “If traditional OCR extracts words, Granite-Docling extracts meaning. That’s a paradigm shift for AI-driven document intelligence.” — Dave Bergmann, Senior AI Writer, IBM

Future Enhancements

IBM’s roadmap includes:

  • Larger Granite-Docling models (512M and 900M parameters).

  • Integration of DocTags into IBM watsonx.ai workflows.

  • Expansion of Docling-eval benchmarking ecosystem.

  • Enhanced multilingual stability.

  • Optimized inference speed for edge deployment.

FAQs

Q1. How does Granite-Docling differ from OCR?
A: OCR extracts plain text, often losing structure. Granite-Docling preserves layout, tables, and contextual relationships via DocTags.

Q2. Is Granite-Docling open source?
A: Yes, it is available on Hugging Face under an Apache 2.0 license.

Q3. Can it handle handwritten content?
A: Current versions are optimized for digital documents; handwriting remains a limitation.

Q4. Does it replace the Docling library?
A: No. Granite-Docling complements Docling pipelines but can also run standalone.

Mermaid Diagram: Granite-Docling Pipeline

ibm-granite-docling-document-conversion-pipeline

Conclusion

Granite-Docling represents a leap forward in document conversion. By combining compact design, layout-preserving intelligence, and DocTags markup, IBM delivers a model suited for enterprise, research, and multilingual contexts. Positioned as the backbone of the Docling ecosystem, Granite-Docling ensures documents are not just digitized but truly understood.

Membership not found