PDFio-based pdftopdf filter bugs

[ValdikSS]

Hey @tillkamppeter, @uddhavphatak, @michaelrsweet

I've compiled the latest libcupsfilters and tested pdftopdf on several PDF files, just a small compatibility and performance test.
I found several bugs while testing.

Tested on:

• libcupsfilters a02c490
• PDFio ae97788

PDF was processed with:

PPD=LBP1120CAPTPPD.ppd FINAL_CONTENT_TYPE=application/vnd.cups-raster \
pdftopdf 1 1 1 1 'PageSize=A4 fitplot' "$file" > "output/${file_noext}_new.pdf"

LBP1120CAPTPPD.ppd.txt

And then converted to JPEG with:

gs -dQUIET -dBATCH -dSAFER -dNOPAUSE -sDEVICE=jpeg -sOutputFile="output/${file_noext}%03d.jpg" -sstdout=%stderr -sPageList=1 -r300 -dDEVICEWIDTHPOINTS=595 -dDEVICEHEIGHTPOINTS=842 -f "$file"

Test 1

File: 4 - Transparency PDF test - Designer.pdf
Result: incorrect transparency handling: the gradient is more transparent, bottom square blending (on the shoulder) is incorrect.

Test 2

File: TransparencyTest.pdf
Result: incorrect transparency handling: the gradient is more transparent (less blue).

Test 3

File: Niere2025a.pdf
Result: crash, output file not created.

…
Key 'N' not found.
ERROR: missing required flags
WARNING: Appearance stream is missing required /BBox.
DEBUG: special case ignore annotation with no appearance
DEBUG: Opening field stream 16/16... with obj number 275
Key 'N' not found.
ERROR: missing required flags
WARNING: Appearance stream is missing required /BBox.
DEBUG: special case ignore annotation with no appearance
DEBUG: Opening content stream 1/3...
DEBUG: Opened stream 1, resmap[3]=(nil)
DEBUG: Opening content stream 2/3...
DEBUG: Opened stream 2, resmap[3]=(nil)
DEBUG: Opening content stream 3/3...
DEBUG: Opened stream 3, resmap[3]=(nil)
DEBUG: page_dict_cb(dict=0xe48d60, key="Font", outpage=0xe9b660), type=5
DEBUG: page_dict_cb(dict=0xe48d60, key="ProcSet", outpage=0xe9b660), type=1
DEBUG: page_dict_cb(dict=0xe48d60, key="XObject", outpage=0xe9b660), type=5
DEBUG: ppdFilterEmitJCL: Filter function (PID 882055) exited with no errors.
ERROR: pdftopdf filter function failed.

Test 4

File: Wendzel2025a.pdf
Result: crash, output file not created.

…
Key 'N' not found.
ERROR: missing required flags
WARNING: Appearance stream is missing required /BBox.
…
DEBUG: page_dict_cb(dict=0x21187c10, key="ProcSet", outpage=0x210965f0), type=1
DEBUG: page_dict_cb(dict=0x21187c10, key="XObject", outpage=0x210965f0), type=5
DEBUG: Opening content stream 1/1...
DEBUG: Opened stream 1, resmap[7]=(nil)
DEBUG: page_dict_cb(dict=0x21196f50, key="ColorSpace", outpage=0x210965f0), type=6
DEBUG: page_dict_cb: objval=0x2104a620(615), arrayval=(nil), dictval=0x21099860
DEBUG: page_dict_cb(dict=0x21196f50, key="ExtGState", outpage=0x210965f0), type=6
DEBUG: page_dict_cb: objval=0x2104a540(613), arrayval=(nil), dictval=0x2109a110
DEBUG: page_dict_cb(dict=0x21196f50, key="Pattern", outpage=0x210965f0), type=6
DEBUG: page_dict_cb: objval=0x2104a5b0(614), arrayval=(nil), dictval=0x210a2500
DEBUG: page_dict_cb(dict=0x21196f50, key="ProcSet", outpage=0x210965f0), type=1
DEBUG: page_dict_cb(dict=0x21196f50, key="XObject", outpage=0x210965f0), type=5
DEBUG: ppdFilterEmitJCL: Filter function (PID 882198) exited with no errors.
ERROR: pdftopdf filter function failed.

Test 5

File: Tai2025a.pdf
Result: entire document is missing pictures.

[uddhavphatak]

I have reproduced the issues, looking into it

[uddhavphatak]

Hi @ValdikSS, I checked the test-1 and test-2. when I tried doing

cat input/'4 - Transparency PDF test - Designer.pdf' | CONTENT_TYPE=application/pdf FINAL_CONTENT_TYPE=application/vnd.cups-raster LD_LIBRARY_PATH=../libcupsfilters/.libs ../../cups-filters/pdftopdf 1 1 1 1 'PageSize=A4 fitplot' > ./output/'1-TransparencyPDFtest-Designer_new.pdf'.pdf

cat input/TransparencyTest.pdf | CONTENT_TYPE=application/pdf FINAL_CONTENT_TYPE=application/vnd.cups-raster LD_LIBRARY_PATH=../libcupsfilters/.libs ../../cups-filters/pdftopdf 1 1 1 1 'PageSize=A4 fitplot' > ./output/2-TransparencyTest_new.pdf

in the PDF output, I am not able to look at the gradient issues in the converted PDF file, but when I convert them to .jpg files, I can see the issue.

[uddhavphatak]

I thought it was an issue with gs, but I tried converting the original PDF to jpg, and I see no issue in them. I understand the issue now, though I don't know how to solve it.. @michaelrsweet @tillkamppeter could you help me with this, what could be wrong

I am looking into the rest of the 3 issues.

[michaelrsweet]

My own quick testing with ipptransform (from libcups):

• "4 - Transparency PDF test - Designer.pdf": Seems to work, no transparency issues I can see.
• "TransparencyTest.pdf": Seems to work, no transparency issues I can see.
• "Niere2025a.pdf": the file seems to have a hex-encoded Unicode string (for the "Subject" metadata) of over 16k in length, and current PDFio limits such strings to 16k (well, 64k of hex data which translates to 16k of UTF-16 Unicode characters). This causes PDFio to throw an error but shouldn't cause a segfault when copying the dictionary.
• "Wendzel2025a.pdf": Same as "Neire2025a.pdf".
• "Tai2025a.pdf": doesn't have the same issue.

So it looks like the transparency issues are in pdftopdf while the crash issues are in PDFio.

[michaelrsweet]

Increasing the hex string buffer limit to 256k allows "Niere2025a.pdf" and "Wendzel2025a.pdf" to be processed.

[michaelrsweet]

Opened this issue in the PDFio project to track a fix for long strings.

[uddhavphatak]

Hi @michaelrsweet , I tested the ouput of ipptransform for the transparency issues, and the bug is still coming

you won't see the transparency issues if you open the PDF through poppler based xpdf, but it is visible if opening through Ghostscript

gs filename.pdf

could this be a bug in ghostscript?

I tested the same for "Tai2025a.pdf", this wasn't happening in ipptransform utility; its a bug in pdftopdf function

[michaelrsweet]

@uddhavphatak I tested the copied PDFs with macOS Preview and Adobe Reader.

I've also added a new "copy" test to the testpdfio program so you can do:

./testpdfio infile.pdf outfile.pdf

for a straight copy of the pages from one PDF to a new one. Aside from exposing the object copy bug for those two files it should reveal any issues with the PDFio object copying code.

[uddhavphatak]

@michaelrsweet, I tested with these renderers as well. I think that this may be an issue with Ghostscripts rendering, because there is no issues with other renderer's but only when opening using Ghostscript's renderer.

[ValdikSS]

The transparency issues indeed seem to affect only Ghostscript, not even MuPDF. Tested on Poppler/Firefox/Chromium as well.

[uddhavphatak]

I have reported an issue in ghostscript bugzilla about this
https://bugs.ghostscript.com/show_bug.cgi?id=709017

[ValdikSS]

Found another file which causes transparency issues with Poppler (only) after pdftopdf PDFio.
The file is slightly broken and produces warnings (and also caused crash in ghostscript opvp)

anketa_3287343.pdf

[ValdikSS]

Same issue with Poppler.

transpose2.pdf

[michaelrsweet]

FWIW, I've found the missing error checks in PDFio that are causing the segfault, and increased the maximum string length to allow those files to be read. This will be part of a 1.6.2 release of PDFio (I'll hold off releasing that for a week or so in case other issues are found...)