Skip Menu |

This queue is for tickets about the Test-Weaken CPAN distribution.

Report information
The Basics
Id: 43855
Status: resolved
Priority: 0/
Queue: Test-Weaken

People
Owner: jkegl [...] cpan.org
Requestors: user42 [...] zip.com.au
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: 3.000000



Subject: suggest object contents callback
Date: Thu, 05 Mar 2009 11:30:52 +1100
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
It'd be good if Test::Weaken had a callback to fetch the contents of an object. In Gtk, sub-widgets or sub-objects aren't in fields of the object hash, only in gtk's structs, which you have to pick out from get_property(), get_children(), etc. And I believe in the various "inside-out" class schemes object property data is not in the object itself, but separate. A callback would make it possible to traverse those things (or the parts one thinks may be subject to leaks). I'm not sure how it would look, maybe sub my_gtk_container_children { my ($obj) = @_; return $obj->isa('Gtk2::Container') && $obj->get_children; } leaks ({ constructor => ..., contents => \&my_gtk_container_children }) You can get the same effect in the constructor by recursing into the objects yourself and returning all the parts. I've been doing that for gtk container children. But leaks() makes exactly such a recursive traversal, if it could be told about extra contents/fields/properties in the objects it visits. As with the ignore option I can imagine multiple contents fetching funcs intended for different classes. So a way to pass different bits could be good. contents => [\&gtk_container_children, \&gobject_properties, \&inside_out_contents, Fetching might often be a single method like get_children() above, so a shorthand giving a class and method could be handy, contents => [ ['Gtk2::Container','get_children'], ...
If I understand this correctly, it's a request to extend Test::Weaken to deal with the situation where data internal (in Test::Weaken's sense of "internal" -- having the same life expectancy) to an object is not reachable by following refs, hashes, arrays, etc. from the base object, but requires certain object methods to be called. Thus: sub { my $base = new BaseObject; return [ $base, $base->other_stuff_here(), $base->and_there(), $base->and_in_yet_another_place(), ]; } I think this requirement may be best answered by having the user create a "meta-constructor" like the above to produce a pseudo-object whose only purpose is to aggregate the object for Test::Weaken. If I understand you correctly, you've discovered this method on your own and are already doing that. This situation is unlike the justification for destructor and ignore options. Those options make it possible to do things which could not be done if they didn't exist. Did I understand this correctly?
Re-reading, I think the central issue might be this: Can Test::Weaken's object traversal logic be leveraged for the contents search, rather than having to traverse the base object twice, once to assemble the object's contents, and a second time to actually do the test? I think the double traversal is the more straight-forward approach. And particularly if we get into all the overhead of callbacks and complex possibilities, I think two straight-forward traversals will almost always be more efficient. I'd certainly expect them to be easier on the programmer, particularly if we are talking about the maintenance programmer.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Sat, 07 Mar 2009 09:17:40 +1100
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > If I understand you correctly, you've discovered this method on your > own and are already doing that.
Yes, I got to this sort of thing ... just looking at container children, not property values yet (some of which I suspect persist, and the occasional one of which may be a circular reference). sub container_children_recursively { my ($widget) = @_; if ($widget->can('get_children')) { return ($widget, map { container_children_recursively($_) } $widget->get_children); } else { return ($widget); } } ... constructor => ... return [ $x, container_children_recursively($x) ]; Show quoted text
> Can Test::Weaken's object traversal logic be leveraged for the > contents search,
Yes. Show quoted text
> rather than > having to traverse the base object twice, once to assemble the object's > contents, and a second time to actually do the test?
leaks() has two good things - it notices duplicates, which is vital for circularities, and important for not re-traversing a thing many times. - it looks into the builtin arrayrefs, hashrefs, maybe some anon coderefs in the future, etc To get the full same effect in one's own constructor, with extra 'contents' worked in, I think you'd end up with an almost complete copy of the leaks() code. I believe it could be much easier to tell leaks() about extra visiting on a per-item basis.
You're convincing me on this one. Also, my mind is being changed by using Test::Weaken myself and discovering how convenient it is to write no-op ignore callbacks (that is, ignore callbacks which always return 1) for scanning and debugging purposes. I'm going to think more about this. Once I take care of the ignore option and get an official release out with that, I'll come up with an interface design. On Fri Mar 06 17:18:07 2009, user42@zip.com.au wrote: coderefs in the future, etc Show quoted text
> To get the full same effect in one's own constructor, with extra > 'contents' worked in, I think you'd end up with an almost complete > copy > of the leaks() code. I believe it could be much easier to tell > leaks() > about extra visiting on a per-item basis.
I've just released 2.001_004 to CPAN. It's a release candidate, and if cpantesters blesses and all looks good, it will become 2.002000.
On Wed Mar 11 00:38:50 2009, JKEGL wrote: Show quoted text
> I've just released 2.001_004 to CPAN. It's a release candidate, and if > cpantesters blesses and all looks good, it will become 2.002000.
I really should have made this an update to the other ticket, since I'm still thinking out design issues on this one. It's the other ticket that will be addressed in 2.002000.
Here's some design notes: I'm thinking of adding to the plumbing interface, which now consists only of new() and test() methods, a scan() method. The scan method would allow ignore-style callbacks to be specified as arguments, and these could be set up, for example, to discover the child sub-objects of inside-out objects. The scan method will be non-destructive, and will not actually run a test --- it will provide the opportunity for a preliminary pass over the test object. This provides a very general interface. For your purpose that interface could be used with callbacks that add objects of interest to an array, and then after the test of the main object, the objects in your array could then also be tested using Test::Weaken or the module of your choice. One complication I think I should mention: The present ignore callback is called both when an object is about to be tracked and when it is about to be followed. This means that for some objects it is called twice, which if it's a simple test makes no difference. (The documentation in 2.002000 has been updated to mention this.) For more complicated applications, like those the scan() method would be used for, the user may need to specify callbacks separately for when an object is about to be tracked; and for the three separate situations where array elements, hash elements and independent objects are about to be followed. I'll add the ability to specify these different callbacks separately to leaks() as well. The 'ignore' option will remain available to specify all four at once with a single routine, but options will be added that allow each of the four kinds of callback can be specified separately.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Tue, 17 Mar 2009 11:30:26 +1100
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > This provides a very general interface. For your purpose that interface > could be used with callbacks that add objects of interest to an array, > and then after the test of the main object, the objects in your array > could then also be tested using Test::Weaken or the module of your choice.
Not sure. Being lazy I think I'd prefer leaks() to do the work :-) Show quoted text
> One complication I think I should mention: The present ignore callback > is called both when an object is about to be tracked and when it is > about to be followed. This means that for some objects it is called > twice, which if it's a simple test makes no difference.
I suppose it's also called more than once if an object occurs more than once in the searched tree. I'm sure only makes a difference if you're abusing the ignore with something that has sloppy side effects. I'd suggest making no guarantees about what how often or in what order ignore calls are made. That'd give flexibility in the implementation, without outlawing abuse for secret nefarious purposes. Show quoted text
> three separate situations where array elements, hash elements and > independent objects are about to be followed.
For myself I haven't felt the need for such distinctions. Actually I'd look to the specific rather than the general, for instance I had a go at ignoring Class::Singleton with sub ignore_singleton { my ($obj) = @_; my $class; return (($class = Scalar::Util::blessed($obj)) && $obj->isa('Class::Singleton') && $class->has_instance && $class->instance == $obj); } (which depends on Class::Singleton 1.4 for the has_instance() method, but that's ok). If it were builtin you could maybe ask for it as something like leaks ({ ignore => [ 'Class::Singleton.instance', \&my_other_ignore, ... ] In fact I'd be sorely tempted to have it ignored by default, and the option be to un-ignore the builtin ignores ... if you see what I mean.
I marked the other ticket resolved as of 2.002000. I'll be moving over the next few days, but I'll print out your email, read it over, and use the opportunity to think my approach to this one.
I've reread our conversation here, which I was pleasantly suprised to find is semi-coherent. :-) Here's a new set of "design notes": 1.) I want to avoid incorporating any smarts about particular classes in Test::Weaken. I also want to avoid incorporating much in the way of knowledge about "inside-out" and other Perl OO philosophies, etc., given the state of flux. Later I may redecide, but for the moment I want just to provide tools and let the user provide the knowledge about particular classes and OO philosophies. 2.) I think your special 'contents' syntax can be provided using wrappers and constructors with minimal inconvenience, given the scan() method I outline below. 3.) The argument to the static scan() method will be a test object (NOT a constructor closure), and callbacks. scan()'s callbacks will be called in list context, with the first element of the array being the boolean which indicates whether to "ignore" the object, and the other elements being added to a list which will be the return value of the scan() method. The scan() method will not alter the test object. scan() is intended to be something that can conveniently be called inside a constructor that is producing a meta-object. 4.) Optionally, I will let the user factor out the various situations where the ignore callback is used, even though this didn't seem useful for you. I have to think someone will end up having to care whether their callback is called zero, one or two times, and on what kinds of object. The unfactored callbacks will be kept -- I'll keep the ignore option of the leaks() method, and the scan() method will also have an 'ignore' option.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Sun, 29 Mar 2009 11:50:43 +1100
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > scan()'s callbacks
Sounds a bit like hard work. Leave it for now, maybe someone else will have other ideas what they might want to do. I still think some builtin understanding of some of the most stable OOPery will end up helpful, perhaps with hook code in those classes rather than Test::Weaken for what things represent sub-components. Show quoted text
> I have to think someone will end up having to care whether > their callback is called zero, one or two times,
You'd hope not :-). Anything meant to be a predicate definitely shouldn't; such code really ought to be side-effect free and call-order independent. But a callback perhaps designed to expose the traversal Test::Weaken makes might be allowed to be tighter, like only once per object, and parent object before child object (though without specifying depth-first or breadth-first), or something like that.
Show quoted text
> Sounds a bit like hard work. Leave it for now, maybe someone else will > have other ideas what they might want to do. I still think some builtin > understanding of some of the most stable OOPery will end up helpful
I'm just not ready to jump into Perl OOPery at this point. There are no major obstacles to someone more expert and/or more opinionated on Perl OOPery building something on top of Test::Weaken, and better them than me. Perhaps with this I can close the ticket. I may go ahead with some of the things I've discussed (a scan() method, and factored callbacks) but on a low priority. I'll wait a few days before closing this ticket, just in case you want to point out something I've missed.
Closing this ticket without changes to the module. I don't want to (at least at this time) embed any knowledge of specific classes or OO philosophies into Test::Weaken. I think those who want this can for the time being achieve their end by creating wrappers and modules which use Test::Weaken. There was discussion of enhancements to Test::Weaken which might be useful to wrappers and derived modules, but nothing was identified as particularly urgent. Ticket closed without any changes to the module.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Tue, 07 Apr 2009 11:41:30 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > specific classes
You don't have to start with specifics, but you should leave this ticket open for per-object contents fetching (where it started). I would use that now for gtk container children and selected sub-objects hiding in properties. Anyone with similar non-array/hash/scalar contents would no doubt want the same. Incidentally the "inside-out" style kinda arises in gtk too with its "boxed" bits which are only a scalar ref as the object, so if you want to hang extra data on a subclassed object you have to do it in a separate table (just a hash, it doesn't have to be full blown inside-out class stuff). Though the only bit like that I've done so far has the extra data semi-shared and so generally persisting beyond each object.
OK. Can you create a test case for the contents option? Again, no need for fancy packaging -- I'll repackage it anyway. And again, please, if you can, no dependencies on non-core modules. You are right, the contents option is pretty general and does not really make assumptions about OO philosophy. Also, the two kinds of argument you suggest for the contents options, a code ref or a ref to an array of code refs, both make sense and I'd like to implement them both. That same starting note which suggests a 'contents' option also mentions a shorthand, which I don't want to implement at this point. Re timing: I've a number of things underway, so I'll be a bit more slow even then usual. Since this was filed first, I plan to work on it before the "descend into tied()" ticket. But since you seem to be the main customer, I'll switch the priority if you wish. I'm marking this as "stalled" while waiting for the test case. On Mon Apr 06 21:43:15 2009, user42@zip.com.au wrote: Show quoted text
> "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > specific classes
> > You don't have to start with specifics, but you should leave this ticket > open for per-object contents fetching (where it started). I would use > that now for gtk container children and selected sub-objects hiding in > properties. Anyone with similar non-array/hash/scalar contents would no > doubt want the same.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Mon, 13 Apr 2009 11:41:56 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > Can you create a test case for the contents option?
Yep. Show quoted text
> array of code refs,
Feel free to defer that, initially. I think it'd be good for combining, but maybe there's a better way. If you do it it could be together with the same for an array of 'ignore' funcs. (Though one would accumulate the returns, the other be a short-circuit "OR".) Show quoted text
> "descend into tied()"
That one may be easier, and it's a core perl feature, if you wanted to ponder it first. The only slightly obscure bit seemed to me the possibility of tied "typeglob" file handle thingies. Either way it may be as easy as pushing tied($$ref) each time onto the pending objects to inspect ...
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Thu, 16 Apr 2009 10:32:53 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
I got to this for a "contents" option test file. The idea would be for the contents function to run in array context so it can return multiple values. In the "moredata" leak case it and its sub-array would make an unfreed_count of 2, I think (exercising recursion into the contents returns).
#!/usr/bin/perl # MyObject has data hidden away from Test::Weaken's normal traversals, in # this case separate %data and %moredata hashes like an "inside-out" object, # or lik you sometimes have to do for extra data in a subclass, or even like # kept only in C code structures. package MyObject; use strict; use warnings; my %data; my %moredata; sub new { my ($class) = @_; my $scalar = 'this is a myobject'; my $self = bless \$scalar, $class; $data{$self+0} = [ 'extra data' ]; $moredata{$self+0} = [ 'more extra data', [ 'with a sub-array too' ] ]; return $self; } sub DESTROY { my ($self) = @_; delete $data{$self+0}; delete $moredata{$self+0}; } sub data { my ($self) = @_; return $data{$self+0}; } sub moredata { my ($self) = @_; return $data{$self+0}; } package main; use strict; use warnings; use Test::Weaken; use Test::More tests => 5; sub myobject_contest_func { my ($obj) = @_; return $obj->data, $obj->moredata; } { my $test = Test::Weaken::leaks ({ constructor => sub { return MyObject->new }, contents => \&myobject_contest_func }); is ($test, undef, 'good weaken of MyObject'); } { my $leak; my $test = Test::Weaken::leaks ({ constructor => sub { my $obj = return MyObject->new; $leak = $obj->data; return $obj; }, contents => \&myobject_contest_func, }); ok ($test, 'leaky "data" detection'); is ($test && $test->unfreed_count, 1, 'one object leaked'); } { my $leak; my $test = Test::Weaken::leaks ({ constructor => sub { my $obj = return MyObject->new; $leak = $obj->moredata; return $obj; }, contents => \&myobject_contest_func, }); ok ($test, 'leaky "moredata" detection'); is ($test && $test->unfreed_count, 2, 'one object leaked'); } exit 0;
I'm at work on this one. Some design notes: 1.) Objects followed and tracked because they are returned from a contents() call will be followed and tracked recursively. That means, for example, all objects followed because they were returned from a contents() callback will have the contents() callback called on them. As usual, cycles will not be allowed to present a problem -- they will be caught and cut short. 2.) Only *FOLLOWED* objects will have the contents() callback called on them. This should present no surprises for objects which are both tracked and followed; for objects which are ignored; and for objects which are followed but not tracked. Objects which are tracked but not followed are a more interesting case. Three kinds of objects are tracked but not followed: CODE, SCALAR and VSTRING. Since the return of the contents() callback is in array context, and CODE objects cannot occur in that context (just references to them), CODE objects are not relevant to this discussion. That leaves SCALAR and VSTRING objects. I note the test case you submitted is a blessed scalar, and falls into one of the categories of objects which won't have the contents() callback called on them because they are not usually followed. Should I call the contents() callback on SCALAR's (and VSTRING's)? I am open to persuasion on this point. My original reasoning for not following SCALAR's is that there is nothing to follow. One could make the argument that with the contents() callback and tied scalars that there now *IS* indeed something to follow. Comments appreciated.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Thu, 23 Apr 2009 08:07:46 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
I wrote: Show quoted text
> > [contents.t]
I had a couple of stray "return"s which would prevent the supposed leaks from leaking!
#!/usr/bin/perl # MyObject has data hidden away from Test::Weaken's normal traversals, in # this case separate %data and %moredata hashes like an "inside-out" object, # or lik you sometimes have to do for extra data in a subclass, or even like # kept only in C code structures. package MyObject; use strict; use warnings; my %data; my %moredata; sub new { my ($class) = @_; my $scalar = 'this is a myobject'; my $self = bless \$scalar, $class; $data{$self+0} = [ 'extra data' ]; $moredata{$self+0} = [ 'more extra data', [ 'with a sub-array too' ] ]; return $self; } sub DESTROY { my ($self) = @_; delete $data{$self+0}; delete $moredata{$self+0}; } sub data { my ($self) = @_; return $data{$self+0}; } sub moredata { my ($self) = @_; return $data{$self+0}; } package main; use strict; use warnings; use Test::Weaken; use Test::More tests => 5; sub myobject_contest_func { my ($obj) = @_; return $obj->data, $obj->moredata; } { my $test = Test::Weaken::leaks ({ constructor => sub { return MyObject->new }, contents => \&myobject_contest_func }); is ($test, undef, 'good weaken of MyObject'); } { my $leak; my $test = Test::Weaken::leaks ({ constructor => sub { my $obj = MyObject->new; $leak = $obj->data; return $obj; }, contents => \&myobject_contest_func, }); ok ($test, 'leaky "data" detection'); is ($test && $test->unfreed_count, 1, 'one object leaked'); } { my $leak; my $test = Test::Weaken::leaks ({ constructor => sub { my $obj = MyObject->new; $leak = $obj->moredata; return $obj; }, contents => \&myobject_contest_func, }); ok ($test, 'leaky "moredata" detection'); is ($test && $test->unfreed_count, 2, 'one object leaked'); } exit 0;
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Thu, 23 Apr 2009 08:45:11 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > for objects which are ignored;
So ignored objects don't get a contents call? Yep that sounds right. Show quoted text
> I note the test case you submitted is a blessed scalar, and falls into > one of the categories of objects which won't have the contents() > callback called on them because they are not usually followed.
Oh. Show quoted text
> Should I > call the contents() callback on SCALAR's (and VSTRING's)?
Err, umm, yes I think so, if I understand the question! :) At any rate my sample is fairly typical of what I had in mind. When a class is only a blessed scalar there's of course no room to put extra data inside it, so you have to do something external (either for a subclass, or for separate application level stuff perhaps). The contents call would let you expose that extra to Test::Weaken ...
I'm at work on this. I'm now writing the docs for a developer's release.
As often, having to state things clearly for the documentation revealed changes that need to be made. The documentation states, and the code assumes, that elements of arrays and values of hashes are ALWAYS freed with the array or hash. Therefore, elements need not be tracked. This assumption is false. The construct @second_refereneces = map { \$_ } @object returns puts reference to every value of @array into @second_references. These second references will survive the destruction of @object, and can be copied, can themselves be referenced, etc., etc. This is a bug, if an obscure one, and I will fix it before the next release. Because this changes the counts of unfreed objects, it can affect code that examines the unfreed objects. Examining the unfreed objects is an useful technique for using Test::Weaken, and a user may be doing this in non-throwaway code. So this can be considered a serious interface change. Therefore the next official release will have a new major release number.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Wed, 03 Jun 2009 08:14:52 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > As often, having to state things clearly for the documentation revealed > changes that need to be made.
It focuses the mind :-) Show quoted text
> @second_refereneces = map { \$_ } @object
Evil! I suppose it's also possible for an xsub to make a mistake with its refs when iterating array elements, along similar lines. Good to catch either way. Show quoted text
> code that examines the unfreed objects.
Existing code ought to be fine with extra unfreed things. If not it was much too fragile to start with ...
I've put a developer's release on CPAN with the fix: 2.003_000 I don't know if this is the release candidate. The documentation is revised and up-to-date, but I want to reread before I consider it final-draft.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Sun, 07 Jun 2009 09:51:56 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > The documentation is revised and up-to-date,
I think the contents callback should be the same as the ignore one. Ie. pass it the object itself, not a "probe" ref to a ref. It'll make life very hard if the two calls are different. I don't think the proberef-to-ref would be wanted usually. If it turns out that it is then you can always add a pair of "ignore_proberef" and "contents_proberef" callbacks later on, for subtle cases. In the callback descriptions you could show the sample call, like &$ignore ($obj); instead of describing it in words. When skimming for a reminder etc it's always easier to see a bit of almost-code than read a description. You should consider giving equal billing to the postprocess-unfreed vs ignore-callback; and to wrapper-array vs contents-callback. After all, "there's more than one way to do it" and personal preference or the nature of the data structures may make each have equally good use. In the bit about globs, I don't think it's true they're alway symbol table entries. I think the Symbol.pm module can make anonymous ones these days, and maybe "do { local *FOO }" likewise, and probably "my" style file handles (other RT ticket) similarly. But maybe a perl expert can confirm that.
Actually the contents and ignore calls ARE orthogonal, it's just that the code snippet for the contents call is wrong. The error in the code snippet is a bug and I'll fix it in the next release, which will have to be another developer's release. Both arguments are and, I think, need to be proberefs. Among the reasons for passing proberefs: 1. Allows for a safe copy of the actual argument to the callback while still giving access to and preserving the actual data. 2. For the ignore callback, the actual data can be anything, including things like arrays and hashes which cannot be arguments -- they have to be passed via a reference. 3. I want the user by default one indirection away from anything in the actual object. I think this is needed in a language which auto-strengthens weak references, autovivifies objects, etc. Users who enjoy life in the fast lane can get to the actual data with a single dereference. Test::Weaken in its internals avoids manipulating the actual data objects. When it gets an object, it immediately creates a probe reference and then deals with the data through that. On Sat Jun 06 19:53:32 2009, user42@zip.com.au wrote: Show quoted text
> "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > The documentation is revised and up-to-date,
> > I think the contents callback should be the same as the ignore one. > Ie. pass it the object itself, not a "probe" ref to a ref. It'll make > life very hard if the two calls are different. > > I don't think the proberef-to-ref would be wanted usually. If it turns > out that it is then you can always add a pair of "ignore_proberef" and > "contents_proberef" callbacks later on, for subtle cases.
Correction: it's the ignore snippet which is wrong and needs to be fixed. On Sun Jun 07 19:33:44 2009, JKEGL wrote: Show quoted text
> Actually the contents and ignore calls ARE orthogonal, it's just that > the code snippet for the contents call is wrong. > > The error in the code snippet is a bug and I'll fix it in the next > release, which will have to be another developer's release. > > Both arguments are and, I think, need to be proberefs. Among the > reasons for passing proberefs: > > 1. Allows for a safe copy of the actual argument to the callback while > still giving access to and preserving the actual data. > > 2. For the ignore callback, the actual data can be anything, including > things like arrays and hashes which cannot be arguments -- they have to > be passed via a reference. > > 3. I want the user by default one indirection away from anything in the > actual object. I think this is needed in a language which > auto-strengthens weak references, autovivifies objects, etc. Users who > enjoy life in the fast lane can get to the actual data with a single > dereference. > > Test::Weaken in its internals avoids manipulating the actual data > objects. When it gets an object, it immediately creates a probe > reference and then deals with the data through that. > > On Sat Jun 06 19:53:32 2009, user42@zip.com.au wrote:
> > "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > > > The documentation is revised and up-to-date,
> > > > I think the contents callback should be the same as the ignore one. > > Ie. pass it the object itself, not a "probe" ref to a ref. It'll make > > life very hard if the two calls are different. > > > > I don't think the proberef-to-ref would be wanted usually. If it turns > > out that it is then you can always add a pair of "ignore_proberef" and > > "contents_proberef" callbacks later on, for subtle cases.
I've just uploaded 2.003_001, and the ignore snippet should be correct in that. On Sun Jun 07 20:45:30 2009, JKEGL wrote: Show quoted text
> Correction: it's the ignore snippet which is wrong and needs to be fixed.
Someday I may rethink how I document the postprocess-unfreed vs. ignore-callback; and wrapper-array vs contents-callback. Right now my thinking is that 1.) Most important is how a hurried Test::Weaken newbie reads the docs. 2.) I assume he will (as I do), when looking for features look first at methods and options. 3.) That means he'll naturally tend to pick ignore-callback over postprocess-unfreed and to pick the contents-callback over wrapper-array. I need to balance this and my deprecatory language does this. In particular, I feel if this errs, it does so in the direction of safety. I've softened and shortened the deprecatory language over the revs, but I'm reluctant to drop it. Show quoted text
> You should consider giving equal billing to the postprocess-unfreed vs > ignore-callback; and to wrapper-array vs contents-callback. After all, > "there's more than one way to do it" and personal preference or the > nature of the data structures may make each have equally good use.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Wed, 10 Jun 2009 11:14:58 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > Correction: it's the ignore snippet which is wrong and needs to be fixed.
Really? The t/ignore.t was working fine, and I've been using its style. The ignore ends up being called with both a ref-to-ref and the plain ref does it? For say a %self instance there's a call \%self which is a proberef to it, but then also a \\%self which is a proberef to any scalar reference found in the tree of things? The first one, \%self is good for oopery, and matches what you've previously documented. Does the second \\%self always arise? Maybe blessed scalars which are array elements don't have it. (Not that I'd be too worried, the first is the desired one from my point of view.) Show quoted text
> 1. Allows for a safe copy of the actual argument to the callback while > still giving access to and preserving the actual data.
I'm still sure you're worrying far too much about that. :-) Show quoted text
> 2. For the ignore callback, the actual data can be anything, including > things like arrays and hashes which cannot be arguments -- they have to > be passed via a reference.
Err, yeah, I think that's what I've been implicitly thinking when it comes to the object ignoring I had in mind. Show quoted text
> auto-strengthens weak references,
That should be only when copying to a new scalar, shouldn't hurt anything normally. Show quoted text
> autovivifies objects,
Might be some scope for trouble there, can't think of anything offhand.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Wed, 10 Jun 2009 11:28:27 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > 1.) Most important is how a hurried Test::Weaken newbie reads the docs.
Umm, no offence, but newbies might be helped by taking a chain-saw to some of the jargon, and the length! :-) I have to confess I still find tracked, followed, persistent, proberef, etc, a bit hard going. The core idea is a good simple thing: look recursively for anything not freed, and user can identify intentionally global things. If that might be expressed both precisely and concisely ... Maybe a "Gory Details" section towards the end could split out implementation notes and details on exactly what is recursed or checked and why.
On Tue Jun 09 21:28:59 2009, user42@zip.com.au wrote: Show quoted text
> Maybe a "Gory Details" section towards the end could split out > implementation notes and details on exactly what is recursed or checked > and why.
This is a good idea, and in fact I've already got an "IMPLEMENTATION SECTION" toward the end where the gory details can go. I tried to keep the intro to the "DESCRIPTION" jargon-free, emphasizing the DWIM nature of the module. I don't think the current rev gets jargon-heavy before the "Data Objects, Blessed Objects and Structures" section. Re-reorganizing the document will, however, have to wait until after this major release.
On Tue Jun 09 21:15:34 2009, user42@zip.com.au wrote: Show quoted text
> "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > Correction: it's the ignore snippet which is wrong and needs to be
fixed. Show quoted text
> > Really? The t/ignore.t was working fine, and I've been using its style. > > The ignore ends up being called with both a ref-to-ref and the plain ref > does it?
I think what was happening is that since the hash itself is an T::W data object, and I always used proberefs, you were getting a proberef to hash anyway as well as a proberef to ref to hash. Instead of telling T::W to ignore the ref to hash, you were telling it to ignore the hash. This amounted to the same thing in practice, so the error in programming the ignore callback was not caught. Similarly, my snippet was part of my test suite, but my test suite did not catch the error. I'll justify the use of proberefs in a separate message.
Let me put aside the other, minor arguments in favor of the "show stopper": Arrays and hashes have to be treated as data objects, not just the references to them. I can, in a way, blame this on you, Kevin, because originally I did assume I could trade only in references, and not deal directly with the hashes and arrays themselves. You found this caused bugs. Fixing those bugs required me to rewrite the T::W internals to track not only refs to arrays and hashes, but the arrays and hashes themselves. The only way to pass arrays and hashes as arguments, is via references. This forces the use of proberefs internally in T::W, in the ignore callback, and in potential future callbacks (which we'd want to be ortogonal with).
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Thu, 11 Jun 2009 07:39:10 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > you were telling it to ignore the hash.
Ah, yep. Show quoted text
> This > amounted to the same thing in practice, so the error in programming the > ignore callback was not caught.
Actually, I think it's not an error, but what I wanted! The hash is the permanent thing with a lifespan extending beyond the testing. If there's a scalar containing a ref to that hash lurking in the tested tree then I think I want that scalar to be gc-ed as normal. That'd be the ref-to-ref case. In fact I think it may be unusual to want a scalar like that to hang around -- only its target. So I suspect the code was right, maybe for the wrong reasons :-).
I've just uploaded 2.003_003, which is a release candidate.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Wed, 17 Jun 2009 09:13:25 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > I've just uploaded 2.003_003, which is a release candidate.
I believe the sample ignore func in the docs is not right, that it has the effect of ignoring the scalar which holds a reference to a MyGlobal object (hash, array, whatever). This means it ignores the object too of course, but I believe ignoring a leak of the scalar isn't the intention. Sample program below with the documented one as "newdoc_ignore()". Using it doesn't report $global_ref as leaked. The "orig_ignore()" however does get that reported, and only @global_obj ignored.
use strict; use warnings; use Test::Weaken; use Data::Dumper; my @global_obj = ('foo'); bless \@global_obj, 'MyGlobal'; my $global_ref = \@global_obj; sub newdoc_ignore { my ($probe) = @_; return unless Scalar::Util::reftype $probe eq 'REF'; my $thing = ${$probe}; return ( Scalar::Util::blessed($thing) && $thing->isa('MyGlobal') ); } sub orig_ignore { my ($ref) = @_; return Scalar::Util::blessed($ref) && $ref->isa('MyGlobal'); } my $oops; my $leaks = Test::Weaken::leaks ({ constructor => sub { return [ \$global_ref ]; }, ignore => \&newdoc_ignore, } ); print Dumper($leaks);
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Wed, 17 Jun 2009 10:00:00 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > I've just uploaded 2.003_003, which is a release candidate.
The double indirection on the contents is not what I wanted please. I'd like it to callback on the objects themselves, not merely on scalars in the tree which refer to them -- per below. As motivation, returning contents based on the scalars is a bit wasteful, as it's quite likely several scalars will refer to the object, but you only really want to report extra contents just once against the object. (I suspect there may be something fishy in the code with objects which are scalar refs, and I'm pretty sure globs deserve callbacks so filehandle objects can be equal citizens, but starting with arrays and hashes will do what I'm interested in.)

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

I have no problem with this change. In previous discussion I think we may have confused two issues. Issue 1: Do I always have to pass the objects via probe_refs. As justified above, the answer is yes. Issue 2: Should the only objects passed to contents() be refs, so that a hash or array can only be accessed via a double indirection. On this I never had a hard and set opinion. If looking into this doesn't produce some kind of complication, your patch will be incorporated in the next release. That will be another developer's release. On Tue Jun 16 20:00:30 2009, user42@zip.com.au wrote: Show quoted text
> "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > I've just uploaded 2.003_003, which is a release candidate.
> > The double indirection on the contents is not what I wanted please. I'd > like it to callback on the objects themselves, not merely on scalars in > the tree which refer to them -- per below. > > As motivation, returning contents based on the scalars is a bit > wasteful, as it's quite likely several scalars will refer to the object, > but you only really want to report extra contents just once against the > object. > > (I suspect there may be something fishy in the code with objects which > are scalar refs, and I'm pretty sure globs deserve callbacks so > filehandle objects can be equal citizens, but starting with arrays and > hashes will do what I'm interested in.) > >
1.) The behavior you describe for the sample ignore func is the behavior I intended. 2.) The question remains, is that the best behavior to show in an example? Since at this point AFAIK you're T::W's biggest power user, I'll defer to your judgement about what behavior serves best for an example. But ... Please justify why ignore the object itself, but catch leaks of the scalar ref to it. It's not that I'll need convincing, but other people may ask me why the example shows what it does, and "Dunno. Ask Kevin." will not sound good as an answer. So I need to get the story down. My guess is that it is something to do with the objects themselves being persistent, while the data structure you are testing for leaks hold references to those objects. Therefore you want the objects ignored, but are worried about leaks of refs to them. On Tue Jun 16 19:14:50 2009, user42@zip.com.au wrote: Show quoted text
> "Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes:
> > > > I've just uploaded 2.003_003, which is a release candidate.
> > I believe the sample ignore func in the docs is not right, that it has > the effect of ignoring the scalar which holds a reference to a MyGlobal > object (hash, array, whatever). This means it ignores the object too of > course, but I believe ignoring a leak of the scalar isn't the intention. > > Sample program below with the documented one as "newdoc_ignore()". > Using it doesn't report $global_ref as leaked. The "orig_ignore()" > however does get that reported, and only @global_obj ignored. > >
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Sat, 20 Jun 2009 08:24:45 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > Therefore you want the objects ignored, > but are worried about leaks of refs to them.
Yes. There may be 2 dozen scalars in the tree containing references to the hash (etc). The hash persists, but the scalars should get garbage collected. (The hash likely persists because there's yet further scalars outside the tree referring to it, but they're of no concern to testing destruction of the tree.) Ditto it's the hash which has extra contents associated with it, rather than any of the scalars referring to it. (Oh, and I feel now the contents callback should be made for all types of things, so for instance even if there's no builtin descent into a glob or print format thingie the callbacks allow user code to add some. I believe that may in fact simplify both the loop in the code and the documentation of it.)
On Fri Jun 19 18:25:10 2009, user42@zip.com.au wrote: Show quoted text
> Oh, and I feel now > the contents callback should be made for all types of things, so for > instance even if there's no builtin descent into a glob or print > format thingie the callbacks allow user code to add some. I believe > that may in fact simplify both the loop in the code and the > documentation of it.
This is an interesting idea. Let me think about it. My first intent was to have the contents callback be orthogonal with following -- if an object is followed, a contents callback is called, otherwise not. But this of course does not allow it to be used as a way for the user to extend following, and that is intriguing. For example, I don't think I can predict in general how GLOB's should be handled, but a particular user with a particular application may have a very precise idea of what he needs.
I just uploaded 2.003_004. I added 2 grafs to the description of contents and ignore, on the idea that other users may get as confused about callback-on-reference/callback-on-object distinction as we have. Kevin, your latest suggestion about extending the contents callback to all objects is not implemented in this release. It does extend the contents callback to arrays and hashes, as per the previous suggestion and your patch. There will probably need to be another developers' release.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Mon, 22 Jun 2009 10:05:15 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > particular user with a particular application
Or experiment with a general descending rule that might later become builtin.
I just uploaded 2.003_005. It has significant simplifications in the internals, based on feedback from perlmonks. On the negative side, that means the internals have changed significantly. The contents callback is called for data objects of all types. As described in the doc (and as you've suggested, Kevin), this can be used to override the default behaviors. For example, a user could add their own code to look inside code objects. This is a release candidate, though realistically, the docs may need a final touchup.
I just uploaded 2.003_006. Quite a few documentation changes, but no changes to the code. This is a release candidate.
I just uploaded 2.003_007. A few, last documentation touch-ups. No changes to the code. This is a release candidate.
Subject: Re: [rt.cpan.org #43855] suggest object contents callback
Date: Fri, 10 Jul 2009 11:57:20 +1000
To: bug-Test-Weaken [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Jeffrey Kegler via RT" <bug-Test-Weaken@rt.cpan.org> writes: Show quoted text
> > I just uploaded 2.003_007.
The 3.000 works for me ... so far :-).
Fixed in 3.000000