Bug #48770 for PAR-Packer: $0 is not working when PAR is used

Mon Aug 17 10:11:04 2009 mianrojas [...] hotmail.com - Ticket created

Subject:	$0 is not working when PAR is used
Date:	Mon, 17 Aug 2009 16:10:40 +0200
To:	<bug-par [...] rt.cpan.org>
From:	Miguel Angel Rojas Gómez <mianrojas [...] hotmail.com>

Hi all, It seems that $0 var is not processed properly inside PAR binaries: I've created just an small script in which you can see the behaviour I'm talking about: # cat test.pl #!/usr/bin/perl print "\$0 : $0\n"; $0 = "xxxxxxxxxxx 1 2 3 4"; print "--\n"; system("ps -ef | grep -v grep | grep $$"); print "--\n"; print "\$0 : $0\n"; exit 0; # ./test.pl A B C $0 : ./test.pl -- root 2992 2806 0 16:53 pts/0 00:00:00 xxxxxxxxxxx 1 2 3 4 <-- output is right. no issues. -- $0 : xxxxxxxxxxx 1 2 3 4 There is nothing wrong here as you can see. But if you "PAR" the script: # pp -o test test.pl # ./test A B C $0 : ./test -- root 3257 2984 0 13:23 pts/1 00:00:00 xxxxxx A B C <-- See that!!! -- $0 : xxxxxxxxxxx 1 2 3 4 It seems that only the first 6 characters are included!!. It seems to be a bug. I'm using perl version 5.10.0 for a Debian lenny distribution with standard packages. The behaviour of $0 has changed and PAR does not suppose to change the behaviour of the program so I consider this bug important. Let me know if you need further details. Thanks. Show quoted text

_________________________________________________________________ ¿Quieres los nuevos emoticonos en 3D? ¡Descárgatelos gratis! http://www.vivelive.com/emoticonos3d/index2.html

Mon Aug 17 15:12:38 2009 schupp [...] argumentum.de - Correspondence added

Subject:	Re: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Mon, 17 Aug 2009 21:12:19 +0200
To:	bug-PAR [...] rt.cpan.org
From:	Roderich Schupp <schupp [...] argumentum.de>

2009/8/17 Miguel Angel Rojas Gómez via RT <bug-PAR@rt.cpan.org>: Show quoted text

> It seems that $0 var is not processed properly inside PAR binaries:

It is. Your notion of "properly" is wrong :) From the Perl source (perl.c): /* Set PL_origalen be the sum of the contiguous argv[] * elements plus the size of the env in case that it is * contiguous with the argv[]. This is used in mg.c:Perl_magic_set() * as the maximum modifiable length of $0. In the worst case * the area we are able to modify is limited to the size of * the original argv[0]. (See below for 'contiguous', though.) Try the following script (essentially your example): print "before \$0=$0\n"; $0 = "12345678901234567890"; print "after \$0=$0\n"; system qw(ps -f), $$; OK, run it: $ perl foo.pl before $0=foo.pl after $0=12345678901234567890 UID PID PPID C STIME TTY STAT TIME CMD roderich 17517 28077 0 21:04 pts/3 S+ 0:00 12345678901234567890 But this only works because most environments are large enough so that the new value of $0 will fit in to the combined contiguous arg+env space. If we shrink the environment to nothing it doesn't work anymore: $ env - perl foo.pl before $0=foo.pl after $0=12345678901234567890 UID PID PPID C STIME TTY STAT TIME CMD roderich 17647 28077 0 21:06 pts/3 S+ 0:00 12345678901 The same happens in a packed executable because it already manipulates $0 (and @ARGV and the environment) before it even gets to execute your script. Afterwards the "combined contiguous" space has shrunk to the size of $0 itself and that's what you observe. Sorry, no cake. Cheers, Roderich

Mon Aug 17 15:12:39 2009 The RT System itself - Status changed from 'new' to 'open'

Tue Aug 18 05:16:49 2009 mianrojas [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Tue, 18 Aug 2009 11:16:28 +0200
To:	<bug-par [...] rt.cpan.org>
From:	Miguel Angel Rojas Gómez <mianrojas [...] hotmail.com>

Hi Roderich, Thanks for your good explanation. It is really appreciated. When I packed my perl files, I discover the program didn't work as expected and I had to dig down for not few days in order to discover $0 was not working properly. As you have shown, you have gone one step further, it seems the issue is related with the shrink of the environment but I think this is something quite weird. The issue I see here is that there is a command (not PAR'ed ) that is working properly, and then, when it is PAR'ed, it has different behaviour, which this is something I see not acceptable. I remember perlcc could change the behaviour of the program because it does not guarantee full compliant, but I didn't read anything about it in PAR. Indeed, I think I see some references saying use PAR if you do not want to see a weird behaviour in your program. PAR should not shrink the environment variable in a way the behaviour of the program is changed (!!). If it does that (I'm pretty sure there could be a good reason for it, but I am not an expert) it should not affect the specification of the program, as it seems it does. My point heres are 2 then: Could this be fixed? If not, could it be a workaround in which this "shrinking" environment in the PAR files could be overcome and the behaviour of the programs are not changed? Thanks Miguel. Show quoted text

_________________________________________________________________ Internet Explorer 8 más sencillo y seguro ¡Descárgatelo gratis! http://events.es.msn.com/noticias/internet-explorer-8/

Tue Aug 18 07:58:53 2009 RSCHUPP [...] cpan.org - Correspondence added

On Tue Aug 18 05:16:49 2009, mianrojas@hotmail.com wrote: Show quoted text

> As you have shown, you have gone one step further, it seems the issue > is related with > the shrink of the environment but I think this is something quite > weird.

It's not so weird if you know how most Unixes (including Linux) implement the passing of arguments + environment to a program (i.e. across execve) and how ps gets its information. Show quoted text

> The issue I see here is that there is a command (not PAR'ed ) > that is working properly, > and then, when it is PAR'ed, it has different behaviour, which this is > something I see not acceptable.

As I've said before, your example "works" by accident (albeit chances are good that it works most of the time). The perlvar man page explains that setting $0 and how this is reflected in ps etc are highly OS dependent and the only thing you could reasonably expect is up to the length of the original $0 (and that's what you observe). In general, it is NOT reasonable to expect that a packed program will behave absoolutely identical to the original script. PAR tries hard and jumps thru several hoops to make some stupid constructs magically work out of the box. But there are known cases, where it doesn't work and you have to change your expectations and your code. Show quoted text

> My point heres are 2 then: > > Could this be fixed? > If not, could it be a workaround in which this "shrinking" environment > in the PAR files could be overcome and the behaviour of the programs > are not changed?

It might be possible to rewrite the handling of argv and envp in PAR::Packer (actually it's parl as embedded in every packed executable) so that the "shrinking" of the environment can be avoided. Patches welcome, the stuff is in myldr/*.c in the PAR::Packer source. You might want to consult the Perl source (mg.c and perl.c in particular) to study the contortions Perl has to go thru to implement the current behaviour.

Tue Aug 18 09:16:24 2009 mianrojas [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Tue, 18 Aug 2009 15:16:02 +0200
To:	<bug-par [...] rt.cpan.org>
From:	Miguel Angel Rojas Gómez <mianrojas [...] hotmail.com>

Hi, Here are my comments: Show quoted text

> In general, it is NOT reasonable to expect that a packed program > will behave absoolutely identical to the original script. PAR tries > hard and jumps thru several hoops to make some stupid constructs > magically work out of the box. But there are known cases, where > it doesn't work and you have to change your expectations and > your code.

Ok, that would mean that PAR a perl file is not really useful if, as you mentioned, because perl code behaviour would change. Unless you really know which functions/contructions are not affected. Is there a list in which we can take a look at? I think this is something should be stated in the man pages and CPAN PAR web. Show quoted text

> It might be possible to rewrite the handling of argv and envp > in PAR::Packer (actually it's parl as embedded in every > packed executable) so that the "shrinking" of the environment > can be avoided.

Show quoted text

> Patches welcome, the stuff is in myldr/*.c in the PAR::Packer source. > You might want to consult the Perl source (mg.c and perl.c in > particular) to study the contortions Perl has to go thru to > implement the current behaviour.

Thanks! I took a look into it and the behaviour seems as you mentioned. But anyway, I think argv and envp behaviour should not be changed in a PAR executable (quite critical if you want to access these option in your code). Of course, it seems this is not something trivial but it is something is impacting perl coding, and if this is not fixed, it should be reflected somewhere. Miguel. Show quoted text

_________________________________________________________________ Entérate de todas las noticias al instante ¡Suscríbete al servicio de Alertas MSN! http://especiales.es.msn.com/noticias/msninforma.aspx

Wed Aug 19 07:37:25 2009 nj88udd02 [...] sneakemail.com - Correspondence added

Subject:	Re: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Wed, 19 Aug 2009 13:37:04 +0200
To:	bug-PAR [...] rt.cpan.org
From:	Steffen Mueller <nj88udd02 [...] sneakemail.com>

Hi Miguel, Miguel Angel Rojas Gómez via RT wrote: Show quoted text

> Queue: PAR > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=48770 >

>> In general, it is NOT reasonable to expect that a packed program >> will behave absoolutely identical to the original script. PAR tries >> hard and jumps thru several hoops to make some stupid constructs >> magically work out of the box. But there are known cases, where >> it doesn't work and you have to change your expectations and >> your code.

> > > > Ok, that would mean that PAR a perl file is not really useful if, as > you mentioned, because perl code behaviour would change. Unless you > really know which functions/contructions are not affected.

You're being a bit silly here. PAR's quite useful and employed for deployment of large applications to many tens of thousands of computers worldwide. Show quoted text

> Is there a list in which we can take a look at? I think this is something > should be stated in the man pages and CPAN PAR web.

There's actually rather few issues. They generally crop up when your code adds assumptions about being able to access the files that a package was loaded from via the file system. OTOH, these things are rarely necessary and usually constitute a code stink anyway. Show quoted text

>> Patches welcome, the stuff is in myldr/*.c in the PAR::Packer source. >> You might want to consult the Perl source (mg.c and perl.c in >> particular) to study the contortions Perl has to go thru to >> implement the current behaviour.

> > > Thanks! I took a look into it and the behaviour seems as you mentioned. > But anyway, I think argv and envp behaviour should not be changed in a PAR > executable (quite critical if you want to access these option in your code). > > Of course, it seems this is not something trivial but it is something is impacting > perl coding, and if this is not fixed, it should be reflected somewhere.

Documentation patches welcome. Cheers, Steffen

Thu Aug 20 06:04:01 2009 mianrojas [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Thu, 20 Aug 2009 12:03:39 +0200
To:	<bug-par [...] rt.cpan.org>
From:	Miguel Angel Rojas Gómez <mianrojas [...] hotmail.com>

Show quoted text

> > > > Ok, that would mean that PAR a perl file is not really useful if, as > > you mentioned, because perl code behaviour would change. Unless you > > really know which functions/contructions are not affected.

> > You're being a bit silly here. PAR's quite useful and employed for > deployment of large applications to many tens of thousands of computers > worldwide.

Please, respect others' opinion as we respect yours ones. You do not need to insult anybody. I also specify "unless". If people is using PAR because they are not impacted by the few issue around you mentioned, that's excellent!!. That would mean PAR is a very good and stable software. Which it does not mean it can not be improved. Show quoted text

> > Is there a list in which we can take a look at? I think this is something > > should be stated in the man pages and CPAN PAR web.

> > There's actually rather few issues. They generally crop up when your > code adds assumptions about being able to access the files that a > package was loaded from via the file system. OTOH, these things are > rarely necessary and usually constitute a code stink anyway. >

As you stated, it seems there are a few issues. If perl code is not using this construction, PAR will be perfect and it is a great application to deploy and to use. What I'm saying here is that perl is using for a lot of scripts by managing and parsing other programs and modifying environment seems to be quite critical for this kind of scripts. That's why I consider this bug as important. Show quoted text

> Documentation patches welcome.

Thanks for the offering. That's really the spirit of free code and that's why I opened this bug. Unfortunally, I am not pretty sure I have the right skills to modify the source code because I'm not an expert on PAR source code. Show quoted text

> > Cheers, > Steffen >

Thanks for your feedback and quick response. Show quoted text

_________________________________________________________________ Internet Explorer 8 más sencillo y seguro ¡Descárgatelo gratis! http://events.es.msn.com/noticias/internet-explorer-8/

Thu Aug 20 08:29:52 2009 angelos [...] unix.gr - Correspondence added

CC:	par [...] perl.org, jesse+par [...] bestpractical.com
Subject:	Re: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Thu, 20 Aug 2009 15:29:26 +0300
To:	bug-PAR [...] rt.cpan.org
From:	Angelos Karageorgiou <angelos [...] unix.gr>

Miguel Angel Rojas Gómez via RT wrote: Show quoted text

> > Please, respect others' opinion as we respect yours ones. You do not need to

I do not think that "Silly" was in insult, so do not feel insulted. It was just a way to tell you that you are dallying too much into something that is a triviality at best. $0 is an OS specific issue that PAR has no control over , if you do not believe me find the source code to sendmail and read the section where it changes its process name to show where the connection to it is coming from. It would be very enlightening for your argument and the list's sake !

Message body is not shown because sender requested not to inline it.

Thu Aug 20 08:37:38 2009 smueller [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #48770] $0 is not working when PAR is used
Date:	Thu, 20 Aug 2009 14:37:14 +0200
To:	bug-PAR [...] rt.cpan.org
From:	Steffen Mueller <smueller [...] cpan.org>

Hi, Angelos Karageorgiou via RT wrote: Show quoted text

> Miguel Angel Rojas Gómez via RT wrote:

>> Please, respect others' opinion as we respect yours ones. You do not need to

> > I do not think that "Silly" was in insult, so do not feel insulted.

thanks for this clarification. Silly was certainly not intended as an insult of any kind. Cheers, Steffen

Thu Aug 20 08:47:23 2009 RSCHUPP [...] cpan.org - Correspondence added

Folks, may I humbly remind you that this a bug tracker, not a discussion forum. Please use the PAR mailing list (par@perl.org) for the latter. Thanx, Roderich

Wed Aug 26 09:38:36 2009 smueller [...] cpan.org - Severity Wishlist added

Wed Aug 26 10:42:31 2009 smueller [...] cpan.org - Queue changed from PAR to PAR-Packer