Bug #55049 for devel-nytprof: Devel-NYTProf-3.01

Sat Feb 27 16:42:58 2010 Tim.Bunce [...] pobox.com - Ticket created

Subject:	Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 25 Feb 2010 18:01:59 +0100
To:	"Tim.Bunce [...] pobox.com" <Tim.Bunce [...] pobox.com>
From:	"Eissfeldt, Heiko" <Tim.Bunce [...] pobox.com>

Hello Tim, I am just doing bug hunting for Devel-NYTProf-3.01_94<http://search.cpan.org/~timb/Devel-NYTProf-3.01_94/> under Windows XP with ActiveState Perl 5.8.4 and want to tell you my finds. Symptom: 'make tests' first dies at t/10-run.t and very often in later tests. This is the call stack at that point: msvcr71.dll!_lock_file(void * pf=0x77c5fce0) Line 236 C msvcr71.dll!fgets(char * string=0x01ce8a24, int count=2048, _iobuf * str=0x77c5fce0) Line 69 + 0x6 C NYTProf.dll!NYTP_gets(NYTP_file_t * ifile=0x01ce5b9c, char * * buffer_p=0x0140fd28, unsigned int * len_p=0x0140fc7c) Line 369 + 0xf C Show quoted text

> NYTProf.dll!load_profile_data_from_stream(sv * cb=0x01ce5b9c) Line 3635 + 0x13 C

NYTProf.dll!XS_Devel__NYTProf__Data_load_profile_data_from_file(interpreter * my_perl=0x00236004, cv * cv=0x01bce6f8) Line 4584 C perl58.dll!28040f9f() The problem seems to be this: load_profile_data_from_stream() uses the 'in' filepointer, which is somehow corrupted. That is 'in' points to a structure NYTP_file_t containing the 'file' pointer pointing to a structure _iobuf which has only NULL pointers - in 0x01ce5b9c {file=0x77c5fce0 NYTP_file_t * - file 0x77c5fce0 _iobuf * + _ptr 0x00000000 <bad Ptr> char * _cnt 0 int + _base 0x00000000 <bad Ptr> char * _flag 1 int _file 3 int _charbuf 0 int _bufsiz 0 int + _tmpfname 0x00000000 <bad Ptr> char * so when NYTP_gets() is called, it crashes. 'in' is set from the return value of NYTP_open(). I have not the faintest idea why such a buggy structure is given back... The file nytprof_10-run.out has this content: NYTProf 3 0 # Perl profile database. Generated by Devel::NYTProf on Thu Feb 25 16:47:11 2010 :basetime=1267112831 :xs_version=3.02 :perl_version=5.8.4 :clock_id=-1 :ticks_per_sec=1000000 :nv_size=8 :PL_perldb=784 :application=- P'ð G÷ß¦áÒAp'ð.Gá¦áÒA Hope that helps, Heiko

Download nytprof_10-run.out
application/octet-stream 251b

Message body not shown because it is not plain text.

Sat Feb 27 17:10:45 2010 Tim.Bunce [...] pobox.com - Correspondence added

CC:	"Eissfeldt, Heiko" <heiko.eissfeldt [...] siemens.com>, develnytprof-dev [...] googlegroups.com
Subject:	Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Sat, 27 Feb 2010 17:09:43 -0500
To:	Tim Bunce via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	Tim Bunce <Tim.Bunce [...] pobox.com>

Hi Heiko. Just a hunch... try editing t/10-run.t like this: --- t/10-run.t (revision 1085) +++ t/10-run.t (working copy) @@ -11,7 +11,7 @@ use Devel::NYTProf::Run qw(profile_this); run_test_group( { - extra_options => { stmts => 0 }, # RT#50851 + extra_options => { stmts => 0, compress => 0 }, # RT#50851 extra_test_count => 1, extra_test_code => sub { my ($profile, $env) = @_; and let us know (ASAP please as this is the last issue holding up a release). If that does _not_ fix it then please: 1. install the module 2. set the NYTPROF env vat to "trace=99" 3. send me the output of: nytprofhtml --file nytprof_10-run.out Thanks. Tim. On Sat, Feb 27, 2010 at 04:42:59PM -0500, Tim Bunce via RT wrote: Show quoted text

> Sat Feb 27 16:42:58 2010: Request 55049 was acted upon. > Transaction: Ticket created by Tim.Bunce@pobox.com > Queue: devel-nytprof > Subject: Devel-NYTProf-3.01_94 under Windows XP > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: Tim.Bunce@pobox.com > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=55049 > > > > Hello Tim, > > I am just doing bug hunting for Devel-NYTProf-3.01_94<http://search.cpan.org/~timb/Devel-NYTProf-3.01_94/> > under Windows XP with ActiveState Perl 5.8.4 and want to tell you my finds. > > Symptom: > 'make tests' first dies at t/10-run.t and very often in later tests. > > This is the call stack at that point: > msvcr71.dll!_lock_file(void * pf=0x77c5fce0) Line 236 C > msvcr71.dll!fgets(char * string=0x01ce8a24, int count=2048, _iobuf * str=0x77c5fce0) Line 69 + 0x6 C > NYTProf.dll!NYTP_gets(NYTP_file_t * ifile=0x01ce5b9c, char * * buffer_p=0x0140fd28, unsigned int * len_p=0x0140fc7c) Line 369 + 0xf C

> > NYTProf.dll!load_profile_data_from_stream(sv * cb=0x01ce5b9c) Line 3635 + 0x13 C

> NYTProf.dll!XS_Devel__NYTProf__Data_load_profile_data_from_file(interpreter * my_perl=0x00236004, cv * cv=0x01bce6f8) Line 4584 C > perl58.dll!28040f9f() > The problem seems to be this: > > load_profile_data_from_stream() uses the 'in' filepointer, which is somehow corrupted. > > That is > 'in' points to a structure NYTP_file_t containing the > 'file' pointer pointing to a structure _iobuf which has only NULL pointers > > > > - in 0x01ce5b9c {file=0x77c5fce0 NYTP_file_t * > - file 0x77c5fce0 _iobuf * > + _ptr 0x00000000 <bad Ptr> char * > _cnt 0 int > + _base 0x00000000 <bad Ptr> char * > _flag 1 int > _file 3 int > _charbuf 0 int > _bufsiz 0 int > + _tmpfname 0x00000000 <bad Ptr> char * > so when NYTP_gets() is called, it crashes. > > 'in' is set from the return value of NYTP_open(). > I have not the faintest idea why such a buggy structure is given back... > > > The file nytprof_10-run.out has this content: > NYTProf 3 0 > # Perl profile database. Generated by Devel::NYTProf on Thu Feb 25 16:47:11 2010 > :basetime=1267112831 > :xs_version=3.02 > :perl_version=5.8.4 > :clock_id=-1 > :ticks_per_sec=1000000 > :nv_size=8 > :PL_perldb=784 > :application=- > P'ð G÷ß¦áÒAp'ð.Gá¦áÒA > > > Hope that helps, > Heiko > >

Mon Mar 01 05:02:55 2010 heiko.eissfeldt [...] siemens.com - Correspondence added

CC:	"develnytprof-dev [...] googlegroups.com" <develnytprof-dev [...] googlegroups.com>
Subject:	AW: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Mon, 1 Mar 2010 10:59:28 +0100
To:	Tim Bunce <Tim.Bunce [...] pobox.com>, Tim Bunce via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	"Eissfeldt, Heiko" <heiko.eissfeldt [...] siemens.com>

Hi Tim, sorry, no changes so far. I changed the test => the app crashed. So I installed the module and ran it like proposed. The app crashed, the debugger points to the same location for the crash as in my previous email. Sorry, no output file is created since the reading fails before. H:\.cpan\build\Devel-NYTProf-3.01_94>nytprofhtml --file t/nytprof_10-run.out # trace=99 Generating report... Reading t/nytprof_10-run.out reading profile data from file t/nytprof_10-run.out <CRASH> ====== Please convince me, that the NYTP_open() function is implemented correctly (that is portably). During debugging I found the call to fopen() in NYTP_open() is made into perl58.dll instead of to the equally named function of the C library. I am sure fopen() returns the buggy structure that is used later in NYTP_gets(). Is PerlIO_open needed here instead? Sorry for the bads news, greetings Heiko Show quoted text

-----Ursprüngliche Nachricht----- Just a hunch... try editing t/10-run.t like this: --- t/10-run.t (revision 1085) +++ t/10-run.t (working copy) @@ -11,7 +11,7 @@ use Devel::NYTProf::Run qw(profile_this); run_test_group( { - extra_options => { stmts => 0 }, # RT#50851 + extra_options => { stmts => 0, compress => 0 }, # RT#50851 extra_test_count => 1, extra_test_code => sub { my ($profile, $env) = @_; and let us know (ASAP please as this is the last issue holding up a release). If that does _not_ fix it then please: 1. install the module 2. set the NYTPROF env vat to "trace=99" 3. send me the output of: nytprofhtml --file nytprof_10-run.out Thanks. Tim. On Sat, Feb 27, 2010 at 04:42:59PM -0500, Tim Bunce via RT wrote:

> I am just doing bug hunting for Devel-NYTProf-3.01_94<http://search.cpan.org/~timb/Devel-NYTProf-3.01_94/> > under Windows XP with ActiveState Perl 5.8.4 and want to tell you my finds. > > Symptom: > 'make tests' first dies at t/10-run.t and very often in later tests. > > This is the call stack at that point: > msvcr71.dll!_lock_file(void * pf=0x77c5fce0) Line 236 C > msvcr71.dll!fgets(char * string=0x01ce8a24, int count=2048, _iobuf * str=0x77c5fce0) Line 69 + 0x6 C > NYTProf.dll!NYTP_gets(NYTP_file_t * ifile=0x01ce5b9c, char * * buffer_p=0x0140fd28, unsigned int * len_p=0x0140fc7c) Line 369 + 0xf C

> > NYTProf.dll!load_profile_data_from_stream(sv * cb=0x01ce5b9c) Line 3635 + 0x13 C

> NYTProf.dll!XS_Devel__NYTProf__Data_load_profile_data_from_file(interpreter * my_perl=0x00236004, cv * cv=0x01bce6f8) Line 4584 C > perl58.dll!28040f9f() > The problem seems to be this: > > load_profile_data_from_stream() uses the 'in' filepointer, which is somehow corrupted. > > That is > 'in' points to a structure NYTP_file_t containing the > 'file' pointer pointing to a structure _iobuf which has only NULL pointers > > > > - in 0x01ce5b9c {file=0x77c5fce0 NYTP_file_t * > - file 0x77c5fce0 _iobuf * > + _ptr 0x00000000 <bad Ptr> char * > _cnt 0 int > + _base 0x00000000 <bad Ptr> char * > _flag 1 int > _file 3 int > _charbuf 0 int > _bufsiz 0 int > + _tmpfname 0x00000000 <bad Ptr> char * > so when NYTP_gets() is called, it crashes. > > 'in' is set from the return value of NYTP_open(). > I have not the faintest idea why such a buggy structure is given back... > > > The file nytprof_10-run.out has this content: > NYTProf 3 0 > # Perl profile database. Generated by Devel::NYTProf on Thu Feb 25 16:47:11 2010 > :basetime=1267112831 > :xs_version=3.02 > :perl_version=5.8.4 > :clock_id=-1 > :ticks_per_sec=1000000 > :nv_size=8 > :PL_perldb=784 > :application=- > P'ð G÷ß¦áÒAp'ð.Gá¦áÒA > > > Hope that helps, > Heiko > >

Mon Mar 01 05:02:57 2010 The RT System itself - Status changed from 'new' to 'open'

Mon Mar 01 10:13:45 2010 heiko.eissfeldt [...] siemens.com - Correspondence added

Subject:	AW: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Mon, 1 Mar 2010 16:06:03 +0100
To:	Tim Bunce <Tim.Bunce [...] pobox.com>, Tim Bunce via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	"Eissfeldt, Heiko" <heiko.eissfeldt [...] siemens.com>

Hi Tim, More info: The preprocessed output of file FileHandle.c (FileHandle.i, see attachment) has this call to fopen() in function NYTP_open(): FILE *raw_file = (*(*Perl_IStdIO_ptr(((PerlInterpreter *)Perl_get_context())))->pOpen)((*Perl_IStdIO_ptr(((PerlInterpreter *)Perl_get_context()))), (name),(mode)); From this original line FILE *raw_file = fopen(name, mode); It does not seem compatible with libc's fopen()... Here is the complete expanded function NYTP_open(): NYTP_file NYTP_open(const char *name, const char *mode) { FILE *raw_file = (*(*Perl_IStdIO_ptr(((PerlInterpreter *)Perl_get_context())))->pOpen)((*Perl_IStdIO_ptr(((PerlInterpreter *)Perl_get_context()))), (name),(mode)); NYTP_file file; if (!raw_file) return 0; (file = (struct NYTP_file_t*)Perl_safesysmalloc((size_t)((1)*sizeof(struct NYTP_file_t)))); file->file = raw_file; return file; } Mit freundlichen Grüßen Heiko Eißfeldt

Download FileHandle.i.gz
application/x-gzip 131.8k

Message body not shown because it is not plain text.

Wed Mar 10 05:52:17 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Hello Tim, I saw, development is progressing fast, so I need to tell you fast about my work. I could fix the crash for 3.01_94 under Windows XP by rewriting FileHandle.xs with the portable Perl IO API (according to the book; see docs perlclib and perlapio). My Perl used PERLIO, btw. Tests are not succeeding yet, alas. But that is a secondary matter. Please compare the attached code against 3.01_94 und consider/review the rewritten code. Thanks Heiko

Subject:

NYTProf.xs

Message body is not shown because it is too large.

Wed Mar 10 05:54:52 2010 heiko [...] hexco.de - Correspondence added

Subject:	Oops, wrong attachment
From:	heiko [...] hexco.de

Oops, the attachment was the wrong one. Here it comes...

Subject:

FileHandle.xs

Message body is not shown because it is too large.

Wed Mar 10 08:33:41 2010 Tim.Bunce [...] pobox.com - Correspondence added

CC:	Tim.Bunce [...] pobox.com, develnytprof-dev [...] googlegroups.com
Subject:	Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Wed, 10 Mar 2010 13:31:43 +0000
To:	Heiko via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	Tim Bunce <Tim.Bunce [...] pobox.com>

On Wed, Mar 10, 2010 at 05:52:20AM -0500, Heiko via RT wrote: Show quoted text

> > I could fix the crash for 3.01_94 under Windows XP > by rewriting FileHandle.xs with the portable Perl IO API > (according to the book; see docs perlclib and perlapio). My Perl used > PERLIO, btw. > > Tests are not succeeding yet, alas. But that is a secondary matter. > > Please compare the attached code against 3.01_94 und consider/review > the rewritten code.

Thanks Heiko [I've attached the diff for the record.] The problem is that really we'd like to reduce perl'isms in the FileHandle.xs I/O code rather than add more. The I/O is simple so shouldn't be too hard to port to windows, in theory. In the course of investigating this did you identify an actual cause? Could it be related to http://perl.markmail.org/thread/2ixdrw25huun2fx5 ? Tim.

Thu Mar 11 05:28:25 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Show quoted text

> In the course of investigating this did you identify an actual cause?

Not really. The mapped fopen() function returns something not appropriate for further use of non PERLAPIO functions... Show quoted text

> Could it be related to

http://perl.markmail.org/thread/2ixdrw25huun2fx5 ? In the widest sense, yes. PERLIO adds another level of indirection for IO Layers as far I understand. This can only work, if used with new functions from PERLAPIO. Alternatively we could block the mappings and use the libc functions directly. That is what I did, and it compiled, and the module did not crash. But how portable that is, I don't know. See attached diff (against 3.0.1_94) for FileHandle.xs

Subject:

FileHandle.xs.patch

--- FileHandle.xs.org 2010-02-21 17:34:28.000000000 +0100 +++ FileHandle.xs 2010-03-11 11:15:54.826391900 +0100 @@ -18,6 +18,18 @@ #define NEED_sv_2pvbyte #include "ppport.h" +/* deliberately throw IO mappings away */ +#undef fgets +#undef fopen +#undef fclose +#undef fread +#undef fwrite +#undef ftell +#undef fprintf +#undef vfprintf +#undef feof +#undef fflush + #ifdef HAS_ZLIB # include <zlib.h> #endif

Thu Mar 11 07:21:25 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Looking better now. Here is the test output.

Subject:

testresults.out.gz

Download testresults.out.gz
application/x-gzip-compressed 1.7k

Message body not shown because it is not plain text.

Thu Mar 11 10:11:16 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Am Do 11. Mär 2010, 07:21:25, heiko@hexco.de schrieb: Show quoted text

> Looking better now. Here is the test output.

rechecked with my latest diff (#undef ...) against 3.10_51: only one test out of 2598 tests failed, good work :-) Z:\.cpan\build\Devel-NYTProf-3.10_51>nmake test Microsoft (R) Program Maintenance Utility, Version 7.10.3077 Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten. Z:\WINNT\Perl\bin\perl.exe -MExtUtils::Command -e "cp" -- blib\arch\auto\Devel\NYTProf\NYTProf.dll blib\lib/Devel/auto/Devel\NYTProf/NYTProf.dll Z:\WINNT\Perl\bin\perl.exe "-MExtUtils::Command::MM" "- e" "test_harness(0, 'blib\lib', 'blib\arch')" t/50-errno.t t/50-errno....ok 3/8 t/50-errno....NOK 4# Failed test '$! should not be altered by assigning fids to previously unprofiled modules (No such file or directory)' # at t/50-errno.t line 31. # got: '2' # expected: '3' t/50-errno....ok 8/8# Looks like you failed 1 test of 8. t/50-errno....dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 4 Failed 1/8 tests, 87.50% okay Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------ ------- t/50-errno.t 1 256 8 1 12.50% 4 Failed 1/1 test scripts, 0.00% okay. 1/8 subtests failed, 87.50% okay. NMAKE : fatal error U1077: 'Z:\WINNT\Perl\bin\perl.exe': Rⁿckgabe- Code '0x2' Stop.

Thu Mar 11 17:15:18 2010 Tim.Bunce [...] pobox.com - Correspondence added

CC:	Tim.Bunce [...] pobox.com, develnytprof-dev [...] googlegroups.com, Jan Dubois <jand [...] activestate.com>
Subject:	Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 11 Mar 2010 22:13:57 +0000
To:	Heiko via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	Tim Bunce <Tim.Bunce [...] pobox.com>

On Thu, Mar 11, 2010 at 05:28:30AM -0500, Heiko via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=55049 > >

> > In the course of investigating this did you identify an actual cause?

> > Not really. The mapped fopen() function returns something not > appropriate for further use of non PERLAPIO functions... >

> > Could it be related to

> http://perl.markmail.org/thread/2ixdrw25huun2fx5 ? > > In the widest sense, yes. > > PERLIO adds another level of indirection for IO Layers as far I > understand. This can only work, if used with new functions from > PERLAPIO. > > Alternatively we could block the mappings and use the libc functions > directly. That is what I did, and it compiled, and the module did not > crash. But how portable that is, I don't know. > > See attached diff (against 3.0.1_94) for FileHandle.xs

Show quoted text

> rechecked with my latest diff (#undef ...) against 3.10_51: > > only one test out of 2598 tests failed, good work :-)

[t/50-errno.t - I'll reply in a seperate email about that one.] Show quoted text

> --- FileHandle.xs.org 2010-02-21 17:34:28.000000000 +0100 > +++ FileHandle.xs 2010-03-11 11:15:54.826391900 +0100 > @@ -18,6 +18,18 @@ > #define NEED_sv_2pvbyte > #include "ppport.h" > > +/* deliberately throw IO mappings away */ > +#undef fgets > +#undef fopen > +#undef fclose > +#undef fread > +#undef fwrite > +#undef ftell > +#undef fprintf > +#undef vfprintf > +#undef feof > +#undef fflush

I'm happy with the approach, but I'd like to wrap it in an #ifdef so it only applies to the affected platforms. What puzzles me is why Jan didn't hit the same problem in http://groups.google.com/group/develnytprof-dev/browse_frm/thread/7ec4850ff164c95b/e241a51712be9efd#e241a51712be9efd You said you're using "Windows XP with ActiveState Perl 5.8.4" Could you post your perl -V output so we get the full details? Jan, could you do the same so we can compare? I want to find an appropriate #ifdef incantation to wrap it with. Any suggestions? Tim.

Thu Mar 11 17:39:51 2010 jand [...] ActiveState.com - Correspondence added

Subject:	RE: [develnytprof-dev: 1825] Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 11 Mar 2010 14:38:52 -0800
To:	<develnytprof-dev [...] googlegroups.com>, "'Heiko via RT'" <bug-devel-nytprof [...] rt.cpan.org>
From:	"Jan Dubois" <jand [...] activestate.com>

On Thu, 11 Mar 2010, Tim Bunce wrote: Show quoted text

> You said you're using "Windows XP with ActiveState Perl 5.8.4" > Could you post your perl -V output so we get the full details? > > Jan, could you do the same so we can compare?

I've attached it at the bottom, but AFAIK there was only a single ActivePerl release for 5.8.4 (build 810), so the output should be identical. Show quoted text

> I want to find an appropriate #ifdef incantation to wrap it with.

As everything worked for me, it seems unlikely that there is any #define that would explain the difference. Things that are still different in Heiko's setup are the compiler version (Visual Studio 2003 I think) and OS (Windows XP). I've been using VC6 and MingW for the compilers and Win2000 for the test VM (but probably not MinGW with 5.8.4 because build 810 doesn't support this without some tweaks). Show quoted text

> Any suggestions?

If anything I would suspect that the compiler version might make a difference (and or the Platform SDK, if installed). But I'm not keen on spending too much time on testing a 7 year old compiler with a 6 year old version of ActivePerl unless the problem can also be reproduced with a current version of Perl. Ok, I have one more idea: When you compile a module with VS2003, then that module links against MSVCR71.dll instead of MSVCRT.dll, so you end up with 2 CRT libraries inside a single process. This normally works fine as long as you don't mix things like memory allocation (call MSVCR71.free() on a pointer returned by MSVCRT.malloc()) and file I/O (call MSVCR71.fputs() on a FILE* returned by MSVCRT.fopen()) etc. All the macro redefinitions in the Perl headers are supposed to protect you from these mixups, but there may be a bug somewhere. I don't have a VM with VS2003, and I don't know if I have a license anywhere anymore anyways, so it is not easy for me to test. Cheers, -Jan Summary of my perl5 (revision 5 version 8 subversion 4) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -NO_HASH_SEED -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX', optimize='-MD -Zi -DNDEBUG -O1', cppflags='-DWIN32' ccversion='', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -libpath:"C: \Perl810\lib\CORE" -machine:x86' libpth=C:\PROGRA~1\MICROS~3\VC98\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib gnulibc_version='undef' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -libpath:"C:\Perl810\lib\CORE" -machine:x86' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_ CONTEXT PERL_IMPLICIT_SYS Locally applied patches: ActivePerl Build 810 22751 Update to Test.pm 1.25 21540 Fix backward-compatibility issues in if.pm Built under MSWin32 Compiled at Jun 1 2004 11:52:21 @INC: c:/Perl810/lib c:/Perl810/site/lib .

Thu Mar 11 17:56:54 2010 Tim.Bunce [...] pobox.com - Correspondence added

CC:	Tim.Bunce [...] pobox.com, develnytprof-dev [...] googlegroups.com
Subject:	Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 11 Mar 2010 22:55:57 +0000
To:	Heiko via RT <bug-devel-nytprof [...] rt.cpan.org>
From:	Tim Bunce <Tim.Bunce [...] pobox.com>

On Thu, Mar 11, 2010 at 10:11:19AM -0500, Heiko via RT wrote: Show quoted text

> > rechecked with my latest diff (#undef ...) against 3.10_51: > > only one test out of 2598 tests failed, good work :-) > > t/50-errno....NOK 4# Failed test '$! should not be altered by > assigning fids to previously unprofiled modules (No such file or > directory)' > # at t/50-errno.t line 31. > # got: '2' > # expected: '3'

I can't reproduce this. The NYTProf.xs code has logic to save and restore errno around the entry points to/from perl. I tried adding code in FileHandle.xs to explicitly set errno in all the i/o functions but still couldn't get test failures (unless I also disabled the errno save/restore logic). I tried with and without zlib. r1166. Is it reliably repeatable for you? To help me narrow down the cause please try the following changes: - edit t/50-errno.t to delete everything after line 31. - change the test count from 8 to 4 - rerun to check it fails the same way - rerun with each of the following changes to $ENV{NYTPROF} (line 6): "start=init:file=$nytprof_out:blocks=0:stmts=0" "start=init:file=$nytprof_out:blocks=0:subs=0" "start=init:file=$nytprof_out:blocks=0:stmts=0:subs=0" - also send me (off-list) the log from running whichever is the last of those which fails, but with ":trace=90" appended to it. Thanks! Tim.

Thu Mar 11 18:50:20 2010 Tim.Bunce [...] pobox.com - Correspondence added

CC:	'Heiko via RT' <bug-devel-nytprof [...] rt.cpan.org>
Subject:	Re: [develnytprof-dev: 1826] Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 11 Mar 2010 23:49:20 +0000
To:	develnytprof-dev [...] googlegroups.com
From:	Tim Bunce <Tim.Bunce [...] pobox.com>

Thanks Jan. For Heiko's config, NYTProf is very broken without the #undefs so I'd like to get those in. Assuming your VS2003 hypothesis is correct, is there a macro to detect that compiler? And would using that macro to wrap the #undefs be a reasonable solution? That leaves the t/50-errno.t failures. I'd be happy enough to skip the failing tests on that system. Is there some way in perl to detect that perl was built with VS2003 and/or the dual libraries? If not then I could just skip the tests on windows systems with old versions of perl, say <= 5.8.9. Tim. On Thu, Mar 11, 2010 at 02:38:52PM -0800, Jan Dubois wrote: Show quoted text

> On Thu, 11 Mar 2010, Tim Bunce wrote:

> > You said you're using "Windows XP with ActiveState Perl 5.8.4" > > Could you post your perl -V output so we get the full details? > > > > Jan, could you do the same so we can compare?

> > I've attached it at the bottom, but AFAIK there was only a single ActivePerl > release for 5.8.4 (build 810), so the output should be identical. >

> > I want to find an appropriate #ifdef incantation to wrap it with.

> > As everything worked for me, it seems unlikely that there is any #define > that would explain the difference. > > Things that are still different in Heiko's setup are the compiler version > (Visual Studio 2003 I think) and OS (Windows XP). I've been using VC6 and MingW for > the compilers and Win2000 for the test VM (but probably not MinGW with 5.8.4 > because build 810 doesn't support this without some tweaks). >

> > Any suggestions?

> > If anything I would suspect that the compiler version might make a difference > (and or the Platform SDK, if installed). But I'm not keen on spending too > much time on testing a 7 year old compiler with a 6 year old version of > ActivePerl unless the problem can also be reproduced with a current version > of Perl. > > Ok, I have one more idea: When you compile a module with VS2003, then that module > links against MSVCR71.dll instead of MSVCRT.dll, so you end up with 2 CRT libraries > inside a single process. This normally works fine as long as you don't mix things > like memory allocation (call MSVCR71.free() on a pointer returned by MSVCRT.malloc()) > and file I/O (call MSVCR71.fputs() on a FILE* returned by MSVCRT.fopen()) etc. > All the macro redefinitions in the Perl headers are supposed to protect you from > these mixups, but there may be a bug somewhere. > > I don't have a VM with VS2003, and I don't know if I have a license anywhere anymore > anyways, so it is not easy for me to test. > > Cheers, > -Jan > > Summary of my perl5 (revision 5 version 8 subversion 4) configuration: > Platform: > osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread > uname='' > config_args='undef' > hint=recommended, useposix=true, d_sigaction=undef > usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define > useperlio=define d_sfio=undef uselargefiles=define usesocks=undef > use64bitint=undef use64bitall=undef uselongdouble=undef > usemymalloc=n, bincompat5005=undef > Compiler: > cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -NO_HASH_SEED > -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX', > optimize='-MD -Zi -DNDEBUG -O1', > cppflags='-DWIN32' > ccversion='', gccversion='', gccosandvers='' > intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 > d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 > ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 > alignbytes=8, prototype=define > Linker and Libraries: > ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -libpath:"C: > \Perl810\lib\CORE" -machine:x86' > libpth=C:\PROGRA~1\MICROS~3\VC98\lib > libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib > netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib > perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib > oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib > libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib > gnulibc_version='undef' > Dynamic Linking: > dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' > cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -libpath:"C:\Perl810\lib\CORE" -machine:x86' > > > Characteristics of this binary (from libperl): > Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_ > CONTEXT PERL_IMPLICIT_SYS > Locally applied patches: > ActivePerl Build 810 > 22751 Update to Test.pm 1.25 > 21540 Fix backward-compatibility issues in if.pm > Built under MSWin32 > Compiled at Jun 1 2004 11:52:21 > @INC: > c:/Perl810/lib > c:/Perl810/site/lib > . > > > -- > You've received this message because you are subscribed to > the Devel::NYTProf Development User group. > > Group hosted at: http://groups.google.com/group/develnytprof-dev > Project hosted at: http://perl-devel-nytprof.googlecode.com > CPAN distribution: http://search.cpan.org/dist/Devel-NYTProf > > To post, email: develnytprof-dev@googlegroups.com > To unsubscribe, email: develnytprof-dev-unsubscribe@googlegroups.com >

Thu Mar 11 22:05:23 2010 jand [...] ActiveState.com - Correspondence added

CC:	"'Heiko via RT'" <bug-devel-nytprof [...] rt.cpan.org>
Subject:	RE: [develnytprof-dev: 1830] Re: [rt.cpan.org #55049] Devel-NYTProf-3.01_94 under Windows XP
Date:	Thu, 11 Mar 2010 19:04:47 -0800
To:	<develnytprof-dev [...] googlegroups.com>
From:	"Jan Dubois" <jand [...] activestate.com>

On Thu, 11 Mar 2010, Tim Bunce wrote: Show quoted text

> > For Heiko's config, NYTProf is very broken without the #undefs > so I'd like to get those in. Assuming your VS2003 hypothesis is correct, > is there a macro to detect that compiler? And would using that macro > to wrap the #undefs be a reasonable solution?

I've verified my "conflicting CRT version" hypothesis now (with VS2008). The issue is indeed a bug in XSUB.h. I've added a workaround for that in r1168. Note that the problem is not specifically about the VS2003 compiler; it is about mixing several different C runtime libraries. The reason I recommend MinGW if you don't have access to VC6 is that it is also using MSVCRT.dll as the runtime library. (MSVCRT.dll is part of Windows itself and not linked to a particular compiler version, which is one of the reasons why ActivePerl continues to be compiled with VC6). Show quoted text

> That leaves the t/50-errno.t failures. I'd be happy enough to > skip the failing tests on that system. Is there some way in perl to > detect that perl was built with VS2003 and/or the dual libraries? > If not then I could just skip the tests on windows systems with old > versions of perl, say <= 5.8.9.

This is again a problem of multiple runtime libraries. The errno you are setting in Devel-NYTProf is not the same that any of the redirected FILE* operations are modifying, so the whole saving/restoring errno doesn't work in this case. errno is not covered at all by the Perl macro redefinition system, so I can't see a way around this. The problem is not related to older Perl versions; it will happen even for 5.12 once it is released. I'm also not convinced that this problem can only happen on Windows with PERL_IMPLICIT_SYS; it should trigger on other platforms too if they have difference C compilers with different runtime libraries. The best I can think of is that you add a little _set_errno() function to set errno to a specific value, and then check if this value propagates back to Perl in the test. Something like: BEGIN { $! = 1; Devel::NYTProf::_set_errno(2); return if $! == 2; print "1..0 # skipped: module linked against different CRT than Perl itself\n"; exit 0; } I'll make a note for myself to look into wrapping errno in the Perl headers once the 5.12 code freeze is over. Cheers, -Jan

Fri Mar 12 03:45:17 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Am Do 11. Mär 2010, 17:15:18, Tim.Bunce@pobox.com schrieb: Show quoted text

> Could you post your perl -V output so we get the full details?

Summary of my perl5 (revision 5 version 8 subversion 4) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 - D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DNO_HASH_SEED -DPERL_IMPLICIT _CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX', optimize='-MD -Zi -DNDEBUG -O1', cppflags='-DWIN32' ccversion='', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf - libpath:"Z:\WINNT\Perl\lib\CORE" -machine:x86' libpth=Z:\PROGRA~1\MICROS~3\VC98\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi 32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib ne tapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib gnulibc_version='undef' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug - opt:ref,icf -libpath:"Z:\WINNT\Perl\lib\CORE" -machine:x86' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS Locally applied patches: ActivePerl Build 810 22751 Update to Test.pm 1.25 21540 Fix backward-compatibility issues in if.pm Built under MSWin32 Compiled at Jun 1 2004 11:52:21 @INC: z:/WINNT/Perl/lib z:/WINNT/Perl/site/lib .

Fri Mar 12 04:00:51 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Am Do 11. Mär 2010, 17:39:51, jand@ActiveState.com schrieb: Show quoted text

> Things that are still different in Heiko's setup are the compiler > version > (Visual Studio 2003 I think) and OS (Windows XP).

Confirmed. I have multiple SDK versions installed, but VS 2003 is active. Show quoted text

> Ok, I have one more idea: When you compile a module with VS2003, then > that module > links against MSVCR71.dll instead of MSVCRT.dll, so you end up with 2 > CRT libraries > inside a single process. This normally works fine as long as you > don't mix things > like memory allocation (call MSVCR71.free() on a pointer returned by > MSVCRT.malloc()) > and file I/O (call MSVCR71.fputs() on a FILE* returned by > MSVCRT.fopen()) etc.

What a mess :-( Show quoted text

> All the macro redefinitions in the Perl headers are supposed to > protect you from > these mixups, but there may be a bug somewhere.

Show quoted text

> I don't have a VM with VS2003, and I don't know if I have a license > anywhere anymore > anyways, so it is not easy for me to test.

@Jan: what do you suggest?

Fri Mar 12 04:27:47 2010 heiko [...] hexco.de - Correspondence added

From:

heiko [...] hexco.de

Show quoted text

> Is it reliably repeatable for you?

Yes, always. Show quoted text

> To help me narrow down the cause please try the following changes: > > - edit t/50-errno.t to delete everything after line 31. > - change the test count from 8 to 4 > - rerun to check it fails the same way

fails like before. Show quoted text

> - rerun with each of the following changes to $ENV{NYTPROF} (line 6): > "start=init:file=$nytprof_out:blocks=0:stmts=0"

succeeds. Show quoted text

> "start=init:file=$nytprof_out:blocks=0:subs=0"

fails. Show quoted text

> "start=init:file=$nytprof_out:blocks=0:stmts=0:subs=0"

succeeds. Show quoted text

> - also send me (off-list) the log from running whichever is the last

of Show quoted text

> those which fails, but with ":trace=90" appended to it.

Done.

Fri Mar 12 11:38:10 2010 TIMB [...] cpan.org - Correspondence added

Fixed in r1168 by Jan.

Fri Mar 12 11:38:22 2010 TIMB [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #55049 for devel-nytprof: Devel-NYTProf-3.01_94 under Windows XP

Preferred bug tracker