community: support advanced text extraction options for pdf documents

[bricefotzo]

Description:

• Updated constructors in PyPDFParser and PyPDFLoader to handle extraction_mode and additional kwargs, aligning with the capabilities of PageObject.extract_text() from pypdf.

• Added test_pypdf_loader_with_layout along with a corresponding example text file to validate layout extraction from PDFs.

Issue: fixes #19735

Dependencies: This change requires updating the pypdf dependency from version 3.4.0 to at least 4.0.0.

Additional changes include the addition of a new test test_pypdf_loader_with_layout and an example text file to ensure the functionality of layout extraction from PDFs aligns with the new capabilities.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2024 8:47pm

[bricefotzo]

Hello Guys!
@ccurme @baskaryan @efriis @maximeperrindev
Hope you're all well!

Just to know that if there is something I can do to help as I saw an activity on last week

[efriis]

is there a way we can make this backwards compatible with version 3 of pypdf?

[bricefotzo]

is there a way we can make this backwards compatible with version 3 of pypdf?

@efriis do you think something like this can be ok?

if pypdf.__version__.startswith('4'):
    page_content = page.extract_text(
        extraction_mode=self.extraction_mode, **self.extraction_kwargs
    )
else:
    page_content = page.extract_text()

[efriis]

Could we reverse it to if pypdf.__version.startswith('3.'): do the old thing, otherwise new? Base assumption being future releases are additive

Will mark this a draft, feel free to mark ready to review when you want me to take a look!

[bricefotzo]

Hi @efriis,
I just added the changes. You can check it. I mentionned you on it.
Feel free to tell me if there is something to change or do better.
Thanks!