Bug #18536 for PAR: PAR, RPMs, ELF

Tue Apr 04 11:24:51 2006 ggb [...] lanl.gov - Ticket created

Subject:	PAR, RPMs, ELF
Date:	Tue, 04 Apr 2006 09:24:23 -0600
To:	bug-par [...] rt.cpan.org
From:	"Giacomo G. Brussino" <ggb [...] lanl.gov>

We have the following bug open with Red Hat: When building rpms that contain packed perl files created with the PAR perl module's pp command the packed perl files do not get a MD5 checksum in the completed RPM package. This causes any attempt to install the affected RPMs normally impossible. Included is a tarball containing the RPM build tree for a perl program called "helloworld" packaged as a pp that has no checksum in the finished RPM. This issue is reproduceable on EL 3 and EL 4. Build tree is from EL 4, 32 bit x86. The missing checksum on the rpm can be viewed with the "rpm -qpilv --dump hw-0.1.i386.rpm" command. The developers at Red Hat are asking the following question: This really sounds like the root cause is poor ELF implementation within PAR if prelink is not working correctly - can anyone confirm that the ELF binaries are 100% spec compliant? It would be nice if you could provide an answer. The official PAR web site has been down for a few weeks already. The current version of PAR is not installing correctly, and it's just hard to get any information... Thank you. Giacomo G. Brussino

Wed May 03 08:36:25 2006 smueller [...] cpan.org - Correspondence added

From:

smueller [...] cpan.org

On Di. 04. Apr. 2006, 11:24:51, ggb@lanl.gov wrote: Show quoted text

> We have the following bug open with Red Hat: > > When building rpms that contain packed perl files created with the PAR > perl module's pp command the packed perl files do not get a MD5 checksum > in the completed RPM package. This causes any attempt to install the > affected RPMs normally impossible. Included is a tarball containing the > RPM build tree for a perl program called "helloworld" packaged as a pp > that has no checksum in the finished RPM. This issue is reproduceable on > EL 3 and EL 4. Build tree is from EL 4, 32 bit x86. The missing checksum > on the rpm can be viewed with the "rpm -qpilv --dump hw-0.1.i386.rpm" > command. > > > The developers at Red Hat are asking the following question: > > This really sounds like the root cause is poor ELF implementation within > PAR if prelink is not working correctly - can anyone confirm that the > ELF binaries are 100% spec compliant? > > > It would be nice if you could provide an answer. > > The official PAR web site has been down for a few weeks already. The > current version of PAR is not installing correctly, and it's just hard > to get any information... > > Thank you. > Giacomo G. Brussino > >

Wed May 03 08:36:25 2006 The RT System itself - Status changed from 'new' to 'open'

Wed May 03 08:37:26 2006 smueller [...] cpan.org - Correspondence added

From:

smueller [...] cpan.org

For reference: This issue was reraised at http://rt.cpan.org/Ticket/Display.html?id=19030

Tue May 23 17:45:56 2006 smueller [...] cpan.org - Correspondence added

There is a workaround that helps when building RPMs that contain binaries created with PAR. You can find it in the FAQ on http://par.wikia.com (last entry as of this writing). HTH, Steffen

Tue May 23 17:45:57 2006 smueller [...] cpan.org - Status changed from 'open' to 'resolved'

Tue May 23 17:45:58 2006 smueller [...] cpan.org - Given to SMUELLER

Mon Jun 19 18:17:37 2006 ggb [...] lanl.gov - Correspondence added

Subject:	Re: [rt.cpan.org #18536] Resolved: PAR, RPMs, ELF
Date:	Mon, 19 Jun 2006 16:16:48 -0600
To:	bug-PAR [...] rt.cpan.org
From:	"Giacomo G. Brussino" <ggb [...] lanl.gov>

On Tue, 2006-05-23 at 17:45 -0400, Steffen Müller via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=18536 > > > According to our records, your request has been resolved. If you have any > further questions or concerns, please respond to this message.

Steffen: The suggested work-around works only for small test cases. It doe not work with larger programs. Here's what I got from Red Hat: We expect to be able to strip binaries for debuginfo - it looks as if PAR does not work with that: [pauln@enki hw-0.1]$ pp helloworld.pl [pauln@enki hw-0.1]$ ./a.out Hello World [pauln@enki hw-0.1]$ strip a.out [pauln@enki hw-0.1]$ ./a.out Usage: ./a.out [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ src.par ] [ program.pl ] ./a.out [ -B|-b ] [-Ooutfile] src.par If you do not want it stripped then do something like chmod the executable in the buildroot in %install, add the appropriate library requires manually and then use %attr(755,root,root) /usr/local/bin/helloworld in %files With regards to the prelink issue it really looks as if prelink and PAR do not play nice together, so, you might want to add PAR binaries to be blacklisted in /etc/prelink.conf in %post. This developer is also getting another developer to look at the binaries to see if he can figure out what is happening. A quick peek using vimdiff makes it look like all the perl code gets stripped out.

Mon Jun 19 18:17:39 2006 The RT System itself - Status changed from 'resolved' to 'open'

Mon Jul 17 08:59:27 2006 kbaxley [...] redhat.com - Correspondence added

From:

kbaxley [...] redhat.com

Some additional observations... I decided to try and do some research on my own in order to try and gain a better understanding of what's happening. In my un-educated opinion it appears that pp doesn't really produce "bogus" ELF binaries, per se. It may be due to how pp puts together executables that causes such headaches with RPM. I took a look at this report and: http://rt.cpan.org/Public/Bug/Display.html?id=19030 19030 seemed to point to some information that was similar in nature to what our developer had said about RPM stripping executables during the build process. In 19030 there was mention that the pp executable file is the combination of a genuine executable (as produced by the linker) with additional items simply appended (and _not_ registered as an ELF section or something). Those appened items apparently get dropped from the output file when RPM runs strip during the build process. In the case here, it looks like a lot of the perl code gets discarded, thus rendering the executable non-functional. Could it be that the perl code is simply appended to the executable? I also played around some with readelf...I, too, didn't see any problems with readelf reading the file in /usr/local/bin/helloworld. Running /usr/local/bin/helloworld after installing the rpm didn't work whenever I tried to invoke it. [kentb@dhcp59-156 ~]# /usr/local/bin/helloworld Usage: /usr/local/bin/helloworld [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ src.par ] [ program.pl ] /usr/local/bin/helloworld [ -B|-b ] [-Ooutfile] src.par I also tried: pp -o hw helloworld.pl and running ./hw *did* work. When I compared the two files (/usr/local/bin/helloworld & hw) using readelf, they were *exactly* the same...so my guess is that the ELF binary itself isn't necessarily "bogus". My guess is that when we run strip during the rpm build process whatever was attached to the pp executable wasn't in the ELF header information and was tossed away? I don't really know if that constitutes bogus ELF vs. how pp packages executables, but, again, this is a totally un-educated guess based on what I've been researching so far. If I'm missing anything please let me know! Thanks!

Mon Jul 17 10:51:21 2006 smueller [...] cpan.org - Correspondence added

Hi, On Mo. 17. Jul. 2006, 08:59:27, kbaxley wrote: Show quoted text

> In 19030 there was mention that the pp executable file > is the combination of a genuine executable (as produced by the linker) > with additional items simply appended (and _not_ registered as an ELF > section or something). Those appened items apparently get dropped from > the output file when RPM runs strip during the build process. In the > case here, it looks like a lot of the perl code gets discarded, thus > rendering the executable non-functional. Could it be that the perl code > is simply appended to the executable?

to my limited understanding, that is exactly what happens. It generates a valid binary executable (basically a perl) and then appends a) perl code for PAR (and Archive::Zip and such) and b) the zipped user code and files. Show quoted text

> Running /usr/local/bin/helloworld after installing the rpm didn't work > whenever I tried to invoke it. > > [kentb@dhcp59-156 ~]# /usr/local/bin/helloworld > Usage: /usr/local/bin/helloworld [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ > src.par ] [ program.pl ] > /usr/local/bin/helloworld [ -B|-b ] [-Ooutfile] src.par

IIRC, the pp-ed binary is a "parl" with a .par (i.e. zip) file concatenated which uses that archive instead of a user specified one. Show quoted text

> I also tried: > > pp -o hw helloworld.pl > > and running ./hw *did* work. > > When I compared the two files (/usr/local/bin/helloworld & hw) using > readelf, they were *exactly* the same...so my guess is that the ELF > binary itself isn't necessarily "bogus". My guess is that when we run > strip during the rpm build process whatever was attached to the pp > executable wasn't in the ELF header information and was tossed away? I > don't really know if that constitutes bogus ELF vs. how pp packages > executables, but, again, this is a totally un-educated guess based on > what I've been researching so far. If I'm missing anything please let > me know! Thanks!

*Exactly* the same? As in bit-for-bit? I doubt that - or else they should both work or both not work. I suppose they have the same binary/executable component, but /usr/local/bin/helloworld lacks the "payload" - the actual "print qq(Hello World\n);" Perl code which was stripped by rpm. The solution to this problem might either be to a) teach pp to register the appended stuff as ELF whatever (I have no idea about Linux binary executable formats) b) prevent rpm from stripping the executables. I suppose b) is easier and is also what Scott suggested in his reply to #19030. Unfortunately, his suggestion does not seem to be the full solution. I'm sorry, but I can't help you any further with this. Fixing pp to produce "strip-safe" binaries is way beyond my understanding of this. If you find a solution one way or another, please, please let me know. Steffen

Fri Jul 21 16:57:19 2006 kbaxley [...] redhat.com - Correspondence added

From:

kbaxley [...] redhat.com

Okay, It appears that we found a solution that works. It like the pp executables are already stripped, so we don't want rpm stripping them again, which, of course, renders them useless. In this case, we added the following lines to the spec file to keep rpm from running the strip process and not produce debuginfo packages: %define __spec_install_post : %define debug_package %{nil} Don't forget to add the ":" character to __spec_install_post as above or this won't work HTH On Mon Jul 17 10:51:21 2006, SMUELLER wrote: Show quoted text

> Hi, > > On Mo. 17. Jul. 2006, 08:59:27, kbaxley wrote:

> > In 19030 there was mention that the pp executable file > > is the combination of a genuine executable (as produced by the linker) > > with additional items simply appended (and _not_ registered as an ELF > > section or something). Those appened items apparently get dropped from > > the output file when RPM runs strip during the build process. In the > > case here, it looks like a lot of the perl code gets discarded, thus > > rendering the executable non-functional. Could it be that the perl code > > is simply appended to the executable?

> > to my limited understanding, that is exactly what happens. It generates > a valid binary executable (basically a perl) and then appends a) perl > code for PAR (and Archive::Zip and such) and b) the zipped user code and > files. >

> > Running /usr/local/bin/helloworld after installing the rpm didn't work > > whenever I tried to invoke it. > > > > [kentb@dhcp59-156 ~]# /usr/local/bin/helloworld > > Usage: /usr/local/bin/helloworld [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ > > src.par ] [ program.pl ] > > /usr/local/bin/helloworld [ -B|-b ] [-Ooutfile] src.par

> > IIRC, the pp-ed binary is a "parl" with a .par (i.e. zip) file > concatenated which uses that archive instead of a user specified one. >

> > I also tried: > > > > pp -o hw helloworld.pl > > > > and running ./hw *did* work. > > > > When I compared the two files (/usr/local/bin/helloworld & hw) using > > readelf, they were *exactly* the same...so my guess is that the ELF > > binary itself isn't necessarily "bogus". My guess is that when we run > > strip during the rpm build process whatever was attached to the pp > > executable wasn't in the ELF header information and was tossed away? I > > don't really know if that constitutes bogus ELF vs. how pp packages > > executables, but, again, this is a totally un-educated guess based on > > what I've been researching so far. If I'm missing anything please let > > me know! Thanks!

> > *Exactly* the same? As in bit-for-bit? I doubt that - or else they > should both work or both not work. I suppose they have the same > binary/executable component, but /usr/local/bin/helloworld lacks the > "payload" - the actual "print qq(Hello World\n);" Perl code which was > stripped by rpm. > > The solution to this problem might either be to > a) teach pp to register the appended stuff as ELF whatever (I have no > idea about Linux binary executable formats) > b) prevent rpm from stripping the executables. > > I suppose b) is easier and is also what Scott suggested in his reply to > #19030. Unfortunately, his suggestion does not seem to be the full

solution. Show quoted text

> > I'm sorry, but I can't help you any further with this. Fixing pp to > produce "strip-safe" binaries is way beyond my understanding of this. > > If you find a solution one way or another, please, please let me know. > > Steffen

Fri Jul 21 18:10:04 2006 smueller [...] cpan.org - Correspondence added

On Fr. 21. Jul. 2006, 16:57:19, kbaxley wrote: Show quoted text

> It appears that we found a solution that works. It like the pp > executables are already stripped, so we don't want rpm stripping them > again, which, of course, renders them useless.

[...] Wow! Thanks for figuring this out. As you might already know, the PAR core developers aren't too familiar with RPMs, so it might have taken us a long time otherwise. I put the solution into the FAQ on the PAR homepage par.perl.org. Thanks again, Steffen

Fri Jul 21 18:10:06 2006 smueller [...] cpan.org - Status changed from 'open' to 'resolved'