Bug #63918 for PDF-API2: String conversion bug (and patch)

Tue Dec 14 17:25:18 2010 fd [...] taz.de - Ticket created

Subject:	String conversion bug (and patch)
Date:	Tue, 14 Dec 2010 23:25:06 +0100 (CET)
To:	bug-PDF-API2 [...] rt.cpan.org
From:	Frank Doepper <fd [...] taz.de>

Hi, there is a bug that gets exposed when PDF::API2 is about to process a Indexed ColorSpace lookup table. Considering e.g. 3\000f\\3\000f which is hex 3300665c330066 as part of the input stream, PDF::API2::Basic::PDF::String::convert changes this to 3\000f\3\000f which gets hex 330066030066, which is wrong and one byte less, when reading in. The patch is: --- /usr/share/perl5/PDF/API2/Basic/PDF/String.pm~ +++ /usr/share/perl5/PDF/API2/Basic/PDF/String.pm @@ -48,7 +48,7 @@ "t" => "\t", "b" => "\b", "f" => "\f", - "\\" => "\\", + "\\" => "\\\\", "(" => "(", ")" => ")" ); perl is 5.10.1, system is debian lenny. best regards, Frank.

Wed Dec 15 05:43:17 2010 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] AutoReply: String conversion bug (and patch)
Date:	Wed, 15 Dec 2010 11:43:07 +0100 (CET)
To:	Bugs in PDF-API2 via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 14.12.10 um 17:25 schrieb Bugs in PDF-API2 via RT: Show quoted text

> Considering e.g. > > 3\000f\\3\000f > > which is hex 3300665c330066 > > as part of the input stream, PDF::API2::Basic::PDF::String::convert > changes this to > > 3\000f\3\000f > > which gets hex 330066030066, which is wrong and one byte less, when > reading in. > > The patch is: > > --- /usr/share/perl5/PDF/API2/Basic/PDF/String.pm~ > +++ /usr/share/perl5/PDF/API2/Basic/PDF/String.pm > @@ -48,7 +48,7 @@ > "t" => "\t", > "b" => "\b", > "f" => "\f", > - "\\" => "\\", > + "\\" => "\\\\", > "(" => "(", > ")" => ")" > );

Unfortunately this patch does not work well, because it results in doubling \\ in another string: 0000005c00000000 -> 0000005c5c00000000 F.

Wed Dec 15 06:24:10 2010 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] AutoReply: String conversion bug (and patch)
Date:	Wed, 15 Dec 2010 12:24:00 +0100 (CET)
To:	Bugs in PDF-API2 via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 15.12.10 um 11:43 schrieb Frank Doepper: Show quoted text

> Unfortunately this patch does not work well, because it results in > doubling \\ in another string: > > 0000005c00000000 -> 0000005c5c00000000

So, the patch has to be: --- lib/PDF/API2/Basic/PDF/String.pm~ +++ lib/PDF/API2/Basic/PDF/String.pm +0100 @@ -129,11 +129,7 @@ { # if we import binary escapes, # let it be hex on output -- fredo - if($str =~ s/\\([nrtbf\\()])/$trans{$1}/ogi) - { - $self->{' ishex'}=1; - } - if($str =~ s/\\([0-7]{1,3})/chr(oct($1))/oeg) + if($str =~ s/\\([nrtbf\\()]|[0-7]{1,3})/$trans{$1}||chr(oct($1))/oegi) { $self->{' ishex'}=1; } regards, F.

Mon Jan 21 11:30:57 2013 steve [...] deefs.net - Correspondence added

Thanks for submitting this bug. I ended up rewriting the convert() method due to a number of other parsing mistakes it was making, and the new version parses both of these cases correctly. It will be included in release 2.021.

Mon Jan 21 11:30:58 2013 The RT System itself - Status changed from 'new' to 'open'

Mon Jan 21 11:30:58 2013 steve [...] deefs.net - Status changed from 'open' to 'patched'

Mon Jan 21 12:34:11 2013 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] String conversion bug (and patch)
Date:	Mon, 21 Jan 2013 18:33:58 +0100 (CET)
To:	Steve Simms via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 21.01.13 um 11:30 schrieb Steve Simms via RT: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=63918 > > > Thanks for submitting this bug. I ended up rewriting the convert() method > due to a number of other parsing mistakes it was making, and the new > version parses both of these cases correctly. > > It will be included in release 2.021.

Thank you. In the meantime I have found another bug regarding a Indexed Colorspace beginning with "[/Indexed/DeviceCMYK 255(<". I attach the patch I am currently applying to the Debian package libpdf-api2-perl (0.73-1) to this mail. I can try to find the real-life test case if you want. Regards, Frank Doepper taz-edv -- xmpp:fd@jabber.taz.de tel:+49-30-25902-307 pgp:0x0C91F566

Message body is not shown because sender requested not to inline it.

Mon Jan 21 12:34:12 2013 The RT System itself - Status changed from 'patched' to 'open'

Mon Jan 21 12:42:00 2013 steve [...] deefs.net - Correspondence added

Show quoted text

> Thank you. In the meantime I have found another bug regarding a Indexed > Colorspace beginning with "[/Indexed/DeviceCMYK 255(<". I attach the

patch Show quoted text

> I am currently applying to the Debian package libpdf-api2-perl (0.73-1)

to Show quoted text

> this mail.

I think the rewrite should have solved that issue as well. There were several places where the literal string handling wasn't lining up with the PDF spec. You can see the change here and try it out for yourself: https://bitbucket.org/ssimms/pdfapi2/commits/29c2bad23cfe974bba919476e9ed9 56c

Mon Jan 21 12:42:01 2013 steve [...] deefs.net - Status changed from 'open' to 'patched'

Wed Feb 20 12:28:31 2013 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] String conversion bug (and patch)
Date:	Wed, 20 Feb 2013 18:27:43 +0100 (CET)
To:	Steve Simms via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Hi Steve. Am 21.01.13 um 12:42 schrieb Steve Simms via RT: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=63918 > >

> > Thank you. In the meantime I have found another bug regarding a > > Indexed Colorspace beginning with "[/Indexed/DeviceCMYK 255(<". I > > attach the patch I am currently applying to the Debian package > > libpdf-api2-perl (0.73-1) to this mail.

> > I think the rewrite should have solved that issue as well. There were > several places where the literal string handling wasn't lining up with > the PDF spec. > > You can see the change here and try it out for yourself: > https://bitbucket.org/ssimms/pdfapi2/commits/29c2bad23cfe974bba919476e9ed956c

Unfortunately it has not. I have taken 2.020 and applied the patch 29c2bad... to it, but, when I use it, it breaks the Indexed Colorspace. I have attached two PDFs, S43-bs3-bln-01.pdf is the original one, S43-bs3-bln-01.seite.pdf is produced with PDF::API2 and has a broken "Indexed Colorspace". Best regards, Frank Doepper taz-edv -- xmpp:fd@jabber.taz.de pgp:0x0C91F566

Download S43-bs3-bln-01.pdf
application/pdf 1.7m

Message body not shown because it is not plain text.

Download S43-bs3-bln-01.seite.pdf
application/pdf 1.5m

Message body not shown because it is not plain text.

Wed Feb 20 12:28:41 2013 The RT System itself - Status changed from 'patched' to 'open'

Mon Feb 25 17:19:46 2013 steve [...] deefs.net - Correspondence added

To clarify, are you saying that the patch fixes the original issue but not the later issue, or are you saying that both of them are still broken? If the original issue is fixed, I'd like to close this ticket, and a new ticket can be created for the later issue.

Tue Feb 26 10:57:16 2013 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] String conversion bug (and patch)
Date:	Tue, 26 Feb 2013 16:56:10 +0100 (CET)
To:	Steve Simms via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 25.02.13 um 17:19 schrieb Steve Simms via RT: Show quoted text

> To clarify, are you saying that the patch fixes the original issue but > not the later issue, or are you saying that both of them are still > broken?

Both of them are still broken. The attachments in my Feb 20 mail illustrate the original issue, and I attach files to illustrate the later issue to this mail: S05-tt5-02.pdf is before, S05-tt5-02.seite.pdf is after processing with PDF::API2. Do you need some minimal code to reproduce the issues? Best regards, Frank Doepper taz-edv -- xmpp:fd@jabber.taz.de pgp:0x0C91F566

Download S05-tt5-02.pdf
application/pdf 2.4m

Message body not shown because it is not plain text.

Download S05-tt5-02.seite.pdf
application/pdf 2.3m

Message body not shown because it is not plain text.

Tue Feb 26 11:19:12 2013 steve [...] deefs.net - Correspondence added

On Tue Feb 26 10:57:16 2013, fd@taz.de wrote: Show quoted text

> Do you need some minimal code to reproduce the issues?

That would be helpful, yes. I don't use that part of the code in my own work.

Tue Feb 26 13:56:26 2013 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] String conversion bug (and patch)
Date:	Tue, 26 Feb 2013 19:56:15 +0100 (CET)
To:	Steve Simms via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 26.02.13 um 11:19 schrieb Steve Simms via RT: Show quoted text

> On Tue Feb 26 10:57:16 2013, fd@taz.de wrote:

> > Do you need some minimal code to reproduce the issues?

> > That would be helpful, yes. I don't use that part of the code in my own > work.

Well, here it is. $ ./pdf-api2-test.pl S05-tt5-02.pdf s.pdf $ xpdf s.pdf Error: Bad Indexed color space (lookup table string too short) Error (24505): Bad image parameters $ ./pdf-api2-test.pl S43-bs3-bln-01.pdf b.pdf $ xpdf b.pdf Error: Bad Indexed color space (lookup table string too short) Error (32985): Bad image parameters Best, Frank

Message body is not shown because sender requested not to inline it.

Tue Feb 25 11:15:19 2014 steve [...] deefs.net - Correspondence added

I'm going to mark this one as resolved with the recent 2.021 release, since it addresses the original bug report. If you can track down where the problem is for the indexed colorspace, please open up a new ticket with some more details. Likewise, if 2.021 broke something that used to work in 2.020 or earlier, please open up a new ticket with an example (as always, if you can point out exactly where in the code the problem is, that'll make it easiest to fix).

Tue Feb 25 11:15:20 2014 steve [...] deefs.net - Status changed from 'open' to 'resolved'

Tue Feb 25 11:15:21 2014 steve [...] deefs.net - Fixed in 2.021 added

Thu Feb 27 11:51:15 2014 fd [...] taz.de - Correspondence added

Subject:	Re: [rt.cpan.org #63918] String conversion bug (and patch)
Date:	Thu, 27 Feb 2014 17:51:03 +0100 (CET)
To:	Steve Simms via RT <bug-PDF-API2 [...] rt.cpan.org>
From:	Frank Doepper <fd [...] taz.de>

Am 25.02.14 um 11:15 schrieb Steve Simms via RT: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=63918 > > > I'm going to mark this one as resolved with the recent 2.021 release, > since it addresses the original bug report. > > If you can track down where the problem is for the indexed colorspace, > please open up a new ticket with some more details. > > Likewise, if 2.021 broke something that used to work in 2.020 or > earlier, please open up a new ticket with an example (as always, if you > can point out exactly where in the code the problem is, that'll make it > easiest to fix).

I suspect a regression, you seem at least to insert a literal linebreak instead of a 0A byte when reading \n in an () encoded Indexed ColorSpace definition. Indexed ColorSpace processing is again broken, as it can easily be proofed by using the test cases which are already attached to this bug. I am not opening a new bug now, because I have my patched 0.73 working here, for years, in production. So, anyway, thank you a lot for this software. Best regards, Frank.