Support a document loader using the MarkItDown library

[Simon-Stone]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Implement a document loader class that uses the MarkItDown library to convert files into Markdown. From the MarkItDown README:

MarkItDown is a utility for converting various files to Markdown (e.g., for indexing, text analysis, etc). It supports:

PDF
PowerPoint
Word
Excel
Images (EXIF metadata and OCR)
Audio (EXIF metadata and speech transcription)
HTML
Text-based formats (CSV, JSON, XML)
ZIP files (iterates over contents)

Motivation

Formatting and layout of documents can convey important context information to the human reader. Using text-based markup languages like Markdown can be very helpful to preserve this information in the text-only representation passed to Large Language Models. The Markitdown library is a great, versatile tool to easily convert different kinds of files into a Markdown representation.

Proposal

I propose a simple MarkitdownLoader class derived from BaseLoader using markitdown.MarkItDown under the hood (composition pattern).

I will try my hand at this and offer a PR soon.

[Simon-Stone]

I've started working on PR #28960 but I'm running into the challenge that MarkItDown requires Python 3.10+, but langchain_community still supports 3.9.

Is there any precedent for how to deal with this? Or am I just SOL until the powers that be decide to drop support for 3.9?

[Simon-Stone]

As I learned when the PR was closed, an integration like this should live in its own package. That's neat, because it will allow it to have different dependencies than the LangChain core packages. So that solves it!