Baseer: The Vision-Language Model from Misraj For Understanding Arabic Documents and Converting them to Markdown

Baseer is a Vision-Language Model (VLM) we developed at Misraj to convert Arabic document images and PDF files into structured text in Markdown format while preserving the structure. It was trained on ~750,000 pages and outperforms all models in the context of the Arabic language in converting documents to text.

To try Baseer, please visit the platform: https://baseerocr.com/

At Misraj, we believe that the future of advanced Arabic models depends on high-quality data and genuine capabilities in understanding Arabic documents exactly as the user sees them. Many books, official documents, and educational materials are still preserved as images or PDF files that are difficult to search or utilize within smart applications. This challenge becomes even more complex when the language is Arabic, with its connected letters, diverse fonts, the presence of diacritics (Tashkeel), and right-to-left writing direction.

Hence, our work on Baseer. We specifically tailored it for the task of reading Arabic documents and converting them into structured text in Markdown format, preserving both the text and the layout. Baseer was built using the Qwen2.5 VL 3B Instruct base model after fine-tuning it on Arabic data prepared specifically for this task.

Available solutions for reading Arabic documents suffer from numerous weaknesses, most notably: treating any document merely as lines of text without understanding the page layout, such as tables, numbering, and bullet points. Furthermore, some fail completely with older fonts or low-quality scanned pages. Additionally, most global OCR systems were originally designed for languages with disconnected letters, like English, causing their performance to degrade when facing connected and overlapping Arabic script.

At Misraj, we wanted to achieve a model that treats the Arabic document as a complete image. It understands headings, paragraphs, and tables. It links the text to its position on the page, and it outputs the document in a structured format suitable for both training language or vision models and practical, real-world use simultaneously.

Baseer is a vision-language model optimized for the task of converting a document image into a structured Markdown representation. We chose Qwen2.5 VL 3B Instruct as a base model after experiments demonstrated its relative superiority in handling Arabic compared to other open-source models. During the fine-tuning phase, we froze the visual component of the model and updated only the linguistic component to maintain general visual capabilities while dedicating language and structure comprehension specifically to Arabic documents.

In training Baseer, we relied on a dataset consisting of 500,000 pairs of matching images and text. 300,000 of these pairs came from synthetic documents we generated via a processing pipeline that starts with high-quality Markdown texts, converts them to HTML, then Word, then PDF, and finally into images. Throughout this pipeline, we varied fonts, page sizes, colors, column counts, and formatting styles. We also applied a wide range of effects to simulate real-world conditions, such as noise, paper yellowing, shadows, and perspective distortion.

The remaining 200,000 pairs came from real documents, including books, magazines, educational materials, and research papers. We selected pages with complex layouts containing tables, images, and footnotes. We utilized an advanced vision-language model to propose the initial text, and then our team manually reviewed a representative sample of this data to ensure the quality of both the text and the structure. All final texts are written in Markdown format, with tables represented in HTML and special tags added for elements like page numbers and watermarks.

In parallel, we at Misraj developed a new evaluation benchmark named *Misraj DocOCR*, which includes 400 images of diverse Arabic documents alongside manually reviewed reference texts.

Baseer's performance was compared against a wide range of systems and large commercial models, most notably Gemini 2.5 Pro, GPT-5, and Azure Document Intelligence, in addition to popular open-source models like Dots.ocr and Nanonets, as well as Arabic-specific models like Qari.

For the evaluation, we used text-specific metrics such as Word Error Rate (WER) and Character Error Rate (CER); structural matching and linguistic fluency metrics, namely BLEU and ChrF; and metrics that measure the model's ability to preserve accurate tables, such as TEDS and MARS. The table below illustrates the comparative performance results we achieved.

The results, as shown in the table above, indicate that the Baseer model achieved the best performance across most metrics, particularly in WER, CER, TEDS, and MARS. It competes very closely with a massive commercial model (Gemini-2.5-Pro) in the BLEU and CHRF benchmarks, proving its superior performance in understanding Arabic documents.

We designed Baseer to be integrated into various applications, such as:

• Smart Archiving and Converting Paper Archives into a Knowledge Base: Empowering organizations to convert thousands of paper records and documents into structured, machine-processable digital text while preserving the logical structure of the content. This facilitates indexing, classification, and retrieval at scale.
• Search, Retrieval, and Advanced Search: Enabling search within scanned PDF files or images as if they were native text files, with support for advanced search capabilities, allowing users to access relevant information quickly and accurately.
• Developing Cognitive Chatbots using RAG Technologies: Supporting the development of smart assistants based on Retrieval-Augmented Generation (RAG) techniques. The model can answer questions based on an understanding of the archived documents' context—not just keywords—while providing precise references to the sources of the answer within the archive.
• Reading and Understanding Multi-Structured Governmental, Legal, and Commercial Documents: Enabling the reading, comprehension, and analysis of government correspondence, legal documents, and official files regardless of their varied structures or layouts, converting them into a Structured Representation usable in analysis, search, and decision-making systems.

Baseer represents a quantum leap in the field of OCR for Arabic documents. Through carefully curated data, an Arabic-specific evaluation benchmark, and an effective fine-tuning strategy, we at Misraj have been able to deliver a model that outperforms many open-source and commercial alternatives, laying a robust foundation for future Arabic projects that rely on document understanding.

*Research Paper Link:*

https://arxiv.org/pdf/2509.18174v1.pdf