Bug #24166 for Module-ScanDeps: scan_deps doesn't show ALL the dependencies

Sun Dec 31 14:53:21 2006 adrianissott [...] hotmail.com - Ticket created

Subject:

scan_deps doesn't show ALL the dependencies

Hi, I've been seeing some problems with ScanDeps where not all the dependencies present are actually included in the hash returned by the subrountine scan_deps. To see this unzip the scandeps_test.zip file provided. It contains: * scandeps_test.pl: running this re-creates the output files. * use_*.pl: inputs to scandeps_test.pl. * output_*.txt: the output of scandeps_test.pl using my environment. My analysis shows the following problems: i) compare output_use_warnings_and_use_strict.txt and output_use_strict_and_use_warnings.txt. These should be the same but they're not: the types of two modules differ between the files. ii) compare output_use_strict_+_warnings.txt and output_use_strict_and_use_warnings.txt. These should be the same but they're not: the used_by attribute of two modules differ between the files. iii) compare output_use_warnings.txt and output_use_strict.txt The files don't have to be identical but: A) the output format for key: strict.pm differs between the files (problem with Data::Dump I believe) B) despite (A) there is a significant difference between the files for strict.pm's output. output_use_warnings.txt shows it being used by Carp/Heavy.pm whereas output_use_strict.txt shows it isn't used by anything. Potentially it could be because Carp/Heavy.pm changes it's dependencies at runtime but I doubt the module would change it's use of strict.pm. C) warnings.pm is shown as using Carp/Heavy.pm by output_use_strict.txt but not by output_use_warnings.txt. iv) In none of the output files does a module have more than one entry in it's used_by attribute. I couldn't see any different problems when comparing other combinations of the files. I believe the problems are cause by the following: 1) The used_by attribute of a module only ever shows the latest dependency identified. I.e. instead of appending the new dependency the latest one overwrites whatever was there before. 2) None of the dependencies from one of the initial files passed into scan_deps to one of the modules are shown. Potentially this is because scan_deps uses "" instead of the file name. This would explain (iii)(B) and (iii)(C) if "" was overwriting (as per (1)) Carp/Heavy.pm. Note I've not checked the code actually does what I've described above rather I've infered it from the results of the testing. I've marked this bug as Critical as there's not much point using the module if either of the two problems above are present. However, if I am right about the causes of the problem then they don't look too hard to fix so hopefully I will be using this module properly in the near future :) FYI, I'm using the following: * Module-ScanDeps [0.70] * perl, v5.8.4 built for MSWin32-x86-multi-thread, Binary build 810 provided by ActiveState Corp. * Windows XP, Version 2002, Service Pack 2 Regards, Adrian

Subject:

scandeps_test.zip

Download scandeps_test.zip
application/x-zip-compressed 6.6k

Message body not shown because it is not plain text.

Tue Jan 02 15:00:50 2007 adrianissott [...] hotmail.com - Correspondence added

From:

adrianissott [...] hotmail.com

On further thought I'm not so sure the problem in part (i) where the type of the dependency changes is caused by the first dependency record being overwritten by subsequent ones. This is because the used_by attribute didn't change just the type. The reason is that if this is the cause of the problem then that means the calling module has two different types of dependency to the same thing and moreover depending on the order of input to scan_deps is read in a different order. This doesn't seem likely so it looks as if either the type is being incorrectly overwriten by some random data or the determination of the dependency type is incorrect :(

Mon Jan 15 23:44:17 2007 EWILHELM [...] cpan.org - Correspondence added

On Sun Dec 31 14:53:21 2006, adriani wrote: Show quoted text

> i) compare > output_use_warnings_and_use_strict.txt > and > output_use_strict_and_use_warnings.txt.

... Show quoted text

> ii) compare > output_use_strict_+_warnings.txt > and > output_use_strict_and_use_warnings.txt.

... Show quoted text

> iii) compare > output_use_warnings.txt > and > output_use_strict.txt

... Show quoted text

> B) despite (A) there is a significant difference between the files for > strict.pm's output. output_use_warnings.txt shows it being used by > Carp/Heavy.pm whereas output_use_strict.txt shows it isn't used by > anything. Potentially it could be because Carp/Heavy.pm changes it's > dependencies at runtime but I doubt the module would change it's use of > strict.pm. > C) warnings.pm is shown as using Carp/Heavy.pm by > output_use_strict.txt but not by output_use_warnings.txt. > > iv) In none of the output files does a module have more than one entry > in it's used_by attribute.

I cannot replicate your results for (i) or (ii). I belive that (iii) is a symptom of (iv). Please look at scan_deps_static() and add_deps() is the ScanDeps.pm source. It appears that the problem is either in add_deps() or in how it is being called from scan_deps_static(). I'm running * Module-ScanDeps [0.71] * perl, v5.8.4 built for i386-linux-thread-multi * debian "Sarge"

Mon Jan 15 23:44:19 2007 The RT System itself - Status changed from 'new' to 'open'

Tue Jan 16 05:38:50 2007 smueller [...] cpan.org - Broken in 0.71 added

Tue Jan 16 05:38:51 2007 smueller [...] cpan.org - Broken in 0.69 deleted

Tue Jan 16 05:38:52 2007 smueller [...] cpan.org - Severity Critical changed to Normal

Sun Jan 21 18:04:55 2007 adrianissott [...] hotmail.com - Correspondence added

From:

adrianissott [...] hotmail.com

Show quoted text

> I cannot replicate your results for (i) or (ii).

Can you post the results you do get? Show quoted text

> It appears that the problem is either in add_deps() or in how it is

being called from scan_deps_static(). I think it could be higher up the call-stack. My current belief is that in scan_deps the recursion on scan_deps_static isn't correctly set-up. The following things strike me as incorrect are: 1) In the line just before scan_deps is defined: my $Keys = 'files|keys|recurse|rv|skip|first|execute|compile|warn_missing'; I think this should be: my $Keys = 'files|Keys|recurse|rv|skip|first|execute|compile|warn_missing'; [Capital K]? However, changing this by itself didn't fix the problem :( 2) In scan_deps @{$args{Keys}} starts off being undefined, whereas @ {$args{files}} is the list of files passed in by the user. My understanding of how the code works is that these two should match up with Keys containing the filename and files containing the full path. When I printed out the corresponding arrays in scan_deps_static, the first time round the arrays don't match but each and every time after that they do. To get round this I added the line: $args{Keys} = $args{files}; just before scan_deps_static is called the first time. (NB if I had more time I would've added the necessary code to strip the directory parts of the path from each entry in 'files') 3) Another capital K is needed in the following: sub scan_deps_static { my ($args) = @_; my ($files, $keys, $recurse, $rv, $skip, $first, $execute, $compile) = @$args{qw( files keys recurse rv skip first execute compile )}; Show quoted text

> I belive that (iii) is a symptom of (iv).

That seemed plausible to me until I saw the result of fixing the issues above. I've attached the last results of my earlier test code in ScanDeps_Testing.zip. I've not completely analysed these new results but: * output_use_strict.txt correctly shows use_strict.pl as only using strict.pm and similarly for output_use_warnings.txt. * Some modules are still apparantly not used by any anything.

Download ScanDeps_Testing.zip
application/x-zip-compressed 6.2k

Message body not shown because it is not plain text.

Sat Apr 14 05:39:22 2007 smueller [...] cpan.org - Correspondence added

Thanks to Adrian's hard work, this bugs is resolved in the current developer release 0.73_01. Thanks, Adrian! Once the dev release has been reasonably tested on a couple of platforms, I'll make a production release. Steffen

Sat Apr 14 05:39:27 2007 smueller [...] cpan.org - Status changed from 'open' to 'resolved'