Skip Menu |

This queue is for tickets about the Class-XSAccessor CPAN distribution.

Report information
The Basics
Id: 82689
Status: open
Priority: 0/
Queue: Class-XSAccessor

People
Owner: Nobody in particular
Requestors: perl [...] toby.ink
Cc: ribasushi [...] leporine.io
AdminCc:

Bug Information
Severity: Normal
Broken in:
  • 1.13
  • 1.14
  • 1.15
  • 1.16
Fixed in: (no value)



Subject: Mutating references to the return values of getters mutates the contents of the hash
You can alter the contents of a blessed hashref using a "getter" method: my $ref = \($object->get_attr); $$ref++; If this could be fixed without imposing a speed penalty, that would be great. If this quirk is caused by an optimization too important to lose, then it should at least be listed under CAVEATS.
Subject: accessors.pl
use strict; use warnings; use Test::More; sub get_pp { $_[0]->{X}; } use Class::XSAccessor getters => { get_xs => 'X' }; { my $o = bless { X=>1 }, 'main'; my $ref = \($o->get_pp); $$ref++; is($o->get_pp, 1); } { my $o = bless { X=>1 }, 'main'; my $ref = \($o->get_xs); $$ref++; is($o->get_xs, 1); } done_testing;
On Mon Jan 14 04:28:30 2013, TOBYINK wrote: Show quoted text
> You can alter the contents of a blessed hashref using a "getter"
method: Show quoted text
> > my $ref = \($object->get_attr); $$ref++; > > If this could be fixed without imposing a speed penalty, that would be > great. > > If this quirk is caused by an optimization too important to lose, then
it Show quoted text
> should at least be listed under CAVEATS.
Sounds like a bug for all but the lvalue accessor implementations. I consider this enough of a bug that performance takes the back seat (if that's a concern at all). Care to submit a test script (or patch to existing test scripts)? I'll do the fixing if you do that. Pull request to https://github.com/tsee/Class-XSAccessor.git would be ideal. Thank you! Best reagrds, Steffen
On 2013-01-14T17:28:01Z, SMUELLER wrote: Show quoted text
> Care to submit a test script (or patch to > existing test scripts)?
The original bug report already includes a test script. It's just called ".pl" rather than ".t" because RT serves up ".t" files with an annoying troff media type in the HTTP headers.
Subject: Re: [rt.cpan.org #82689] Mutating references to the return values of getters mutates the contents of the hash
Date: Wed, 16 Jan 2013 07:53:03 +0100
To: bug-Class-XSAccessor [...] rt.cpan.org
From: Steffen Mueller <smueller [...] cpan.org>
On 01/15/2013 02:12 PM, Toby Inkster via RT wrote: Show quoted text
> Queue: Class-XSAccessor > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=82689 > > > On 2013-01-14T17:28:01Z, SMUELLER wrote:
>> Care to submit a test script (or patch to >> existing test scripts)?
> > The original bug report already includes a test script. It's just called > ".pl" rather than ".t" because RT serves up ".t" files with an annoying > troff media type in the HTTP headers.
Sorry, my email client hid that from me rather effectively. It's committed to the repo now. I'll work on a fix as soon as I can (no implied time scale). --Steffen
Subject: Re: [rt.cpan.org #82689] Mutating references to the return values of getters mutates the contents of the hash
Date: Wed, 16 Jan 2013 08:03:57 +0100
To: bug-Class-XSAccessor [...] rt.cpan.org
From: Steffen Mueller <smueller [...] cpan.org>
On 01/15/2013 02:12 PM, Toby Inkster via RT wrote: Show quoted text
> Queue: Class-XSAccessor > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=82689 > > > On 2013-01-14T17:28:01Z, SMUELLER wrote:
>> Care to submit a test script (or patch to >> existing test scripts)?
> > The original bug report already includes a test script. It's just called > ".pl" rather than ".t" because RT serves up ".t" files with an annoying > troff media type in the HTTP headers.
Hmm, I have a fix (sv_mortalcopy), but it slows simple accessors down by a bit more than 10%. Ouch. Will have to ponder. --Steffen
On Wed Jan 16 02:04:20 2013, SMUELLER wrote: Show quoted text
> On 01/15/2013 02:12 PM, Toby Inkster via RT wrote:
> > Queue: Class-XSAccessor > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=82689 > > > > > On 2013-01-14T17:28:01Z, SMUELLER wrote:
> >> Care to submit a test script (or patch to > >> existing test scripts)?
> > > > The original bug report already includes a test script. It's just
called Show quoted text
> > ".pl" rather than ".t" because RT serves up ".t" files with an
annoying Show quoted text
> > troff media type in the HTTP headers.
> > Hmm, I have a fix (sv_mortalcopy), but it slows simple accessors down
by Show quoted text
> a bit more than 10%. Ouch. Will have to ponder.
I'm not sure why I never caught this problem in the first place. It's not a complex-to-understand bug after all. Unfortunately, it's seems to me that it's hard to impossible to fix without the aforementioned performance penalty (roughly going from 3.8M calls/s to 3.4M calls/s on my laptop running in powersave mode). Even after careful examination of the XSAccessor code and the relevant portions of perl's sv.c, I can't see a good way around making a copy of the SV and putting that on the save stack (making it a mortal SV). What happens in the current behaviour is that the actual scalar that's stored in the object is put on the stack. The \ operator in your example will create an RV that points at that very SV. What you get is effectively a read-write alias (kind of like what happens in @_). Alas, where in @_, this behaviour is widely apprehended as a valid optimization, I guess it's not something that we can repeat for return values. Chocolateboy, how do you feel about this? I fear sv_mortalcopy it is. Apart from having to fuzz with the savestack, it also incurs a get magic invocation on the scalar that's about to be copied. I'm 95% certain that we can't do without that in the general case. Best regards, Steffen
Show quoted text
> Chocolateboy, how do you feel about this?
If it's required for correctness, then I guess we have no choice.
Might it be possible to have a strict mode and a fast mode (toggled via environment variable or something), with strict mode the default. Authors would run in strict mode, to ensure they're not relying on this bug; users could switch on fast mode in production environments if they need to eek out a little more performance.
On Thu Jan 17 03:21:45 2013, CHOCOLATE wrote: Show quoted text
> > Chocolateboy, how do you feel about this?
> > If it's required for correctness, then I guess we have no choice. >
Would it be an option to only incur the penalty on ro-type accessors (i.e. ro attributes are properly protected) and have the "whatever" current behavior on rw accessors?
On Wed Jan 23 06:07:53 2013, RIBASUSHI wrote: Show quoted text
> On Thu Jan 17 03:21:45 2013, CHOCOLATE wrote:
> > > Chocolateboy, how do you feel about this?
> > > > If it's required for correctness, then I guess we have no choice. > >
> > Would it be an option to only incur the penalty on ro-type accessors > (i.e. ro attributes are properly protected) and have the "whatever" > current behavior on rw accessors?
But you still don't want this to work: $ perl -e 'package F; use Class::XSAccessor {accessors => "foo"}; package main; my $f = bless({foo=>1}, "F");warn ${\$f->foo}++ for 1..10;' 1 at -e line 1. 2 at -e line 1. 3 at -e line 1. 4 at -e line 1. 5 at -e line 1. 6 at -e line 1. 7 at -e line 1. 8 at -e line 1. 9 at -e line 1. 10 at -e line 1. You'd want it closer to this: $ perl -e 'package F;sub foo {$_[0]->{foo} = $_[1] if @_ > 1;return $_[0]->{foo}} package main; my $f = bless({foo=>1}, "F");warn ${\$f- Show quoted text
>foo}++ for 1..10;'
1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. 1 at -e line 1. In other words, I don't see the distinction between desired behaviour of r and rw accessors. I'm happy to be convinced otherwise since I am no fan of the performance penalty. --Steffen
On Fri Jan 25 01:27:19 2013, SMUELLER wrote: Show quoted text
> On Wed Jan 23 06:07:53 2013, RIBASUSHI wrote:
> > On Thu Jan 17 03:21:45 2013, CHOCOLATE wrote:
> > > > Chocolateboy, how do you feel about this?
> > > > > > If it's required for correctness, then I guess we have no choice. > > >
> > > > Would it be an option to only incur the penalty on ro-type accessors > > (i.e. ro attributes are properly protected) and have the "whatever" > > current behavior on rw accessors?
> > But you still don't want this to work:
No you don't. But on a rw accessor it *feels* less catastrophic. Just my gut feeling anyway. Show quoted text
> I'm happy to be convinced otherwise since I am no > fan of the performance penalty.
Given that Mouse is the closest thing to a contender CXSA has[1], and moreover a 10% slowdown will move Mouse to the leading position, I felt that outsourcing our problem is in order [2] ;) Let's see where this goes [1] https://metacpan.org/module/Class::Accessor::Grouped#Benchmark [2] https://rt.cpan.org/Public/Bug/Display.html?id=82945
On Fri Jan 25 04:00:35 2013, RIBASUSHI wrote: Show quoted text
> Given that Mouse is the closest thing to a contender CXSA has[1], and > moreover a 10% slowdown will move Mouse to the leading position, I felt > that outsourcing our problem is in order [2] ;) > > Let's see where this goes > > [1] https://metacpan.org/module/Class::Accessor::Grouped#Benchmark > [2] https://rt.cpan.org/Public/Bug/Display.html?id=82945
So we got a reply finally: https://rt.cpan.org/Public/Bug/Display.html?id=82945#txn-1186243 What does C::XSA do now? :) Is it possible to make this thing switchable at C::XSA *require* time with an environment variable or somesuch? I.e. ability to say "I know that my project and all my dependencies will not take references to the return values, go faster please".
On Wed Jan 16 02:04:20 2013, SMUELLER wrote: Show quoted text
> Hmm, I have a fix (sv_mortalcopy), but it slows simple accessors down by > a bit more than 10%. Ouch. Will have to ponder.
Would it be an option to flip the RO flag on scalars (and only scalars) returned by this?
On Wed Feb 27 09:43:12 2013, RIBASUSHI wrote: Show quoted text
> On Wed Jan 16 02:04:20 2013, SMUELLER wrote:
> > Hmm, I have a fix (sv_mortalcopy), but it slows simple accessors down by > > a bit more than 10%. Ouch. Will have to ponder.
> > Would it be an option to flip the RO flag on scalars (and only scalars) > returned by this?
Sorry for not replying to this earlier. It slipped through the net somehow. No, it's not possible to do that since the scalars returned by the methods are the exact same scalars that are stored in the object. That is the root of the problem. I have, however, spent more time thinking about this in the meantime. The original workaround/fix for the issue was to allocate a new SV, copy the original SV into it, then push the new SV on the mortal stack, then return it. That's slow. My money is on the allocation and mortalization being by far the slowest part usually. This can most likely be improved on considerably by using a PADTMP entry like many other OPs do: It remains allocated, it doesn't need mortalization. Just an SV copy into it and push it on the stack. Downside: It'll be cozy with the core and chances are, it isn't doable with strictly API calls. Secondly, we know it's certain operations where this is a problem at all. my $x = $o->foo; is safe because the SV will be copied into the PAD SV that is $x. \$o->foo is unsafe because the reference will refer to the actual same SV. The latter case can be identified at OP-optimization time by looking at the entersub's op_next execution order pointer something like this: if (OP_CLASS(o->op_next) == OP_REF) { ... do not optimize ... } I didn't bother looking up OP_REF, though. This has a similar cozy-with-core downside, and additionally, it may yield false negatives: It'll be hard to catch all cases. Here's another example with aliasing semantics where I wouldn't know how to detect it statically (and efficiently so): perl -e 'package F; use Class::XSAccessor getters => [qw(foo)]; $x = bless({foo => "bar"} => "F"); sub f {return \@_} $y=f($x->foo); $y->[0]++; warn $x->{foo};' bas at -e line 1. Best regards, Steffen
On Thu Apr 04 00:47:25 2013, SMUELLER wrote: Show quoted text
> On Wed Feb 27 09:43:12 2013, RIBASUSHI wrote:
> > On Wed Jan 16 02:04:20 2013, SMUELLER wrote:
> > > Hmm, I have a fix (sv_mortalcopy), but it slows simple accessors
> down by
> > > a bit more than 10%. Ouch. Will have to ponder.
> > > > Would it be an option to flip the RO flag on scalars (and only
> scalars)
> > returned by this?
> > Sorry for not replying to this earlier. It slipped through the net > somehow. > > No, it's not possible to do that since the scalars returned by the > methods are the exact same scalars that are stored in the object. > That is the root of the problem. > > I have, however, spent more time thinking about this in the meantime. > The original workaround/fix for the issue was to allocate a new SV, > copy the original SV into it, then push the new SV on the mortal > stack, then return it. That's slow. My money is on the allocation > and mortalization being by far the slowest part usually. > > This can most likely be improved on considerably by using a PADTMP > entry like many other OPs do: It remains allocated, it doesn't need > mortalization. Just an SV copy into it and push it on the stack. > Downside: It'll be cozy with the core and chances are, it isn't > doable with strictly API calls. > > Secondly, we know it's certain operations where this is a problem at > all. > > my $x = $o->foo; > > is safe because the SV will be copied into the PAD SV that is $x. > > \$o->foo > > is unsafe because the reference will refer to the actual same SV. > > The latter case can be identified at OP-optimization time by looking > at the entersub's op_next execution order pointer something like > this: > > if (OP_CLASS(o->op_next) == OP_REF) { > ... do not optimize ... > } > > I didn't bother looking up OP_REF, though. > > This has a similar cozy-with-core downside, and additionally, it may > yield false negatives: It'll be hard to catch all cases. Here's > another example with aliasing semantics where I wouldn't know how > to detect it statically (and efficiently so): > > perl -e 'package F; use Class::XSAccessor getters => [qw(foo)]; $x = > bless({foo => "bar"} => "F"); sub f {return \@_} $y=f($x->foo); $y-
> >[0]++; warn $x->{foo};'
> bas at -e line 1.
I've implemented the TARG & sv_setsv based approach in the master branch. No release yet. In my simplistic benchmark, it gives (ro) accessors about a 3% slowdown rather than the 10% with the mortal copy. There is no release yet because I would really, really like to have this code reviewed. chocolateboy, are you up for that? At the same time, I found a bug in the lvalue version of the array accessors. They were installing the non-lvalue version which incidentally worked because we were returning the internal SV pointer. Hilarious :) --Steffen
CC: mail [...] tobyinkster.co.uk
Subject: Re: [rt.cpan.org #82689] Mutating references to the return values of getters mutates the contents of the hash
Date: Fri, 5 Apr 2013 19:17:47 +1100
To: Steffen Mueller via RT <bug-Class-XSAccessor [...] rt.cpan.org>
From: Peter Rabbitson <ribasushi [...] cpan.org>
On Fri, Apr 05, 2013 at 04:11:31AM -0400, Steffen Mueller via RT wrote: Show quoted text
> > I've implemented the TARG & sv_setsv based approach in the master > branch. No release yet. In my simplistic benchmark, it gives (ro) > accessors about a 3% slowdown rather than the 10% with the mortal > copy.
3% is way within reasonable to sacrifice for correctness. I should have a better benchmark(er) this weekend and will share the results. Awesome work Steffen ;) You can also mention in the relnotes (in passing) that this release fixes a bug present in many XS accessor makers, e.g. Mouse ;PPP Cheers
Show quoted text
> I would really, really like to have this code reviewed. > chocolateboy, are you up for that?
It looks fine to me, but bear in mind I have no idea what's it doing or what this discussion is about :-)
It looks fine to me; I too have no idea what's it doing, but at least I know what this discussion is about. :-) That it makes the test case pass is good enough for me.