Bug #97075 for Finance-InteractiveBrokers-SWIG: Segfault/free() error upon exit after contractDetails event

Tue Jul 08 20:46:19 2014 gdg [...] zplane.com - Ticket created

Subject:	Segfault/free() error upon exit after contractDetails event
Date:	Tue, 8 Jul 2014 18:36:15 -0600
To:	bug-Finance-InteractiveBrokers-SWIG [...] rt.cpan.org
From:	Glenn Golden <gdg [...] zplane.com>

Hi Jason, Recently started playing around with the contractDetails() event for the first time, and am running into an intermittent segfault/free() error that was never seen prior to installing the code that does the contractDetails() requests and services the associated events. The contractDetails() event itself seems to work fine: Upon receiving the event, am able to interrogate the returned data structure using the various swig_xxxx_get() methods. All such data looks fine, nothing unusual. And my client continues normally indefinitely after the event. However, on roughly half the occasions when I later ^C out of the client and my SIGINT handler runs, I sometimes see a segfault or a free() error, the backtrace of which seems to suggest that something didn't go quite right in destroying the ContractDetails object. (See attached "segfault.txt".) Of course it's entirely possible it's user braindamage on my part too, perhaps not cleaning up properly or something...? The SIGINT handler seems benign; all it does is whine to stderr that SIGINT was received (and that report does appear on stderr, prior to the system segfault barfage) and then does eDisconnect() on the IBAPI object, followed by exit(0). The event handler for contractDetails() is also trivial at this time; it does nothing other than swig_xxxx_get() one of the values from the object, then prints the value, that's all. I even tried eliminating the get() call, to see if that was possibly related to the segfault. No joy. And with the get() call commented out, the event handler is essentially doing nothing other than sucking three parms from @_. Let me know if you have any ideas on this. If not I'll try to gen up a minimal example for you. Version info: perl 5.20.0, F::IB::SWIG 0.13. Thanks, Glenn

Message body is not shown because sender requested not to inline it.

Wed Jul 09 18:47:47 2014 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #97075] AutoReply: Segfault/free() error upon exit after contractDetails event
Date:	Wed, 9 Jul 2014 16:47:30 -0600
To:	bug-Finance-InteractiveBrokers-SWIG [...] rt.cpan.org
From:	Glenn Golden <gdg [...] zplane.com>

Hi Jason, Found another way to cause a similar failure which may be easier for you to reproduce/diagnose. This method is 100% reproducible on my setup, whereas the exit() issue was intermittent. The event handler (inline below) for contractDetails intentionally issues a call to a non-existent method on the object, whereupon the error message Call dispatch for contractDetails failed with: Can't locate object method "swig_nonExistentMethod_get" via package "Finance::InteractiveBrokers::SWIG::IBAPI::ContractDetails" at [...] issues to stderr, and it then crashes with a free() 'invalid pointer' error. The backtrace is attached. I have no hard evidence that the underlying source of the free() error is the same here as in the originally reported issue which occurs upon exit(), but seems like a not-unreasonable guess. The good thing is that unlike the exit() issue, which is intermittent, this fails consistently every time, at least on my setup. sub contractDetails { my ($self, $rid, $condet) = @_; $t = $condet->swig_tradingHours_get(); # Works ok printf(" MOH: %s\n", $t); $t = $condet->swig_nonExistentMethod_get(); # Segfaults printf(" EV mult: %f\n", $t); } Version info: perl: 5.20.0 F::IB::SWIG: 0.13 Regards, Glenn

Message body is not shown because sender requested not to inline it.

Fri Jul 11 10:36:00 2014 INFIDEL [...] cpan.org - Correspondence added

I think you're running into a double-free() from by the rather ghetto linkage between SWIG and Perl; you may need to use some combination of $condet->DISOWN()/$condet->ACQUIRE(). More info: http://www.swig.org/Doc2.0/Perl5.html#Perl5_nn42 Also, you're using your signal handler a bit unsafely; it's better to do the very minimum possible. See perldoc perlipc's Signals section. Finally, keep in mind this is a really low-level interface to IBAPI; it's best handled by being wrapped in a higher-level facade that handle this for you (as you can see by the countless autogenerated getters/setters).

Fri Jul 11 10:36:00 2014 The RT System itself - Status changed from 'new' to 'open'

Fri Jul 11 10:57:05 2014 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #97075] Segfault/free() error upon exit after contractDetails event
Date:	Fri, 11 Jul 2014 08:56:46 -0600
To:	Jason McManus via RT <bug-Finance-InteractiveBrokers-SWIG [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

Jason McManus via RT <bug-Finance-InteractiveBrokers-SWIG@rt.cpan.org> [2014-07-11 10:36:00 -0400]: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=97075 > > > I think you're running into a double-free() from by the rather ghetto > linkage between SWIG and Perl; you may need to use some combination > of $condet->DISOWN()/$condet->ACQUIRE(). > > More info: > http://www.swig.org/Doc2.0/Perl5.html#Perl5_nn42 >

OK, will look into that, thanks. Show quoted text

> > Also, you're using your signal handler a bit unsafely; it's better to do > the very minimum possible. See perldoc perlipc's Signals section. >

Misunderstanding due to my wording in the ticket, sorry. The signal handler subroutine does nothing other than setting a global variable to show that the SIGINT occurred. The main() function itself examines the global during the request/event interrogation loop and performs the exit() if that global is set. So the problem that occurs upon exit() is not due to the exit() being issued from the asynchronous signal handler context, if that's what you're getting at. Show quoted text

> > Finally, keep in mind this is a really low-level interface to IBAPI; > it's best handled by being wrapped in a higher-level facade that > handle this for you (as you can see by the countless autogenerated > getters/setters). >

Yes, that higher-level facade is exactly what I'm writing. Again, I may have confused the issue by referring to it as a "client", but it's really just an intermediate-level data collection daemon that sits in a loop issuing TWS/IB requests based on UI actions, and then placing event-generated results into a database for the actual trading platorm to interrogate. Anyway, let me look into DISOWN() and ACQUIRE() and see what I can learn there. Thanks, Glenn

Fri Jul 11 21:18:09 2014 gdg [...] zplane.com - Correspondence added

Subject:	Re: [rt.cpan.org #97075] Segfault/free() error upon exit after contractDetails event
Date:	Fri, 11 Jul 2014 19:17:47 -0600
To:	Jason McManus via RT <bug-Finance-InteractiveBrokers-SWIG [...] rt.cpan.org>
From:	Glenn Golden <gdg [...] zplane.com>

Hi Jason, Jason McManus via RT <bug-Finance-InteractiveBrokers-SWIG@rt.cpan.org> [2014-07-11 10:36:00 -0400]: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=97075 > > > I think you're running into a double-free() from by the rather ghetto > linkage between SWIG and Perl; you may need to use some combination of > $condet->DISOWN()/$condet->ACQUIRE(). > > More info: > http://www.swig.org/Doc2.0/Perl5.html#Perl5_nn42 >

Took at look at that SWIG reference. From my understanding of it, there does not seem to be any need for an ACQUIRE() on the contractDetails object, given the way it was obtained and the atomic way in which I'm using it within the event handler, i.e. just extracting its data via a bunch of xxx_get() methods, then done with it forever. But DISOWN()ing it (after copying out its data) does seem to make sense as a requirement for avoiding destructor double free() issues, just as you said. And indeed, doing OISOWN() does seem to avoid the intermittent segfault upon exit. So thanks for that pointer, very useful. Might even be worthwhile to add a quickie note about this to the F::IB::SWIG doc or to the event handler example source when you have time. However, the issue of segfaulting upon calling a non-existent SWIG method is not addressed by this. Minor issue really, perhaps not even worth your time even tracking down or fixing. But if you want, I'll be happy to file a separate ticket on it, just for your bookkeeping purposes, since it seems now sufficiently ununrelated to the original issue to warrant a separate report. Up to you, just let me know. Thanks again for your time and assistance, Glenn

Sat Jul 12 01:46:05 2014 INFIDEL [...] cpan.org - Correspondence added

Show quoted text

> But DISOWN()ing it (after copying out its data) does seem to make > sense as a > requirement for avoiding destructor double free() issues, just as you > said. > And indeed, doing OISOWN() does seem to avoid the intermittent > segfault upon > exit.

I think the root cause is that the object is instantiated in the underlying C++ layer, but then Perl grabs ahold of it again; then, upon shutdown, both Perl and the C++ layer are calling delete() on it (C++ in IBAPI_wrap.cxx:_wrap_delete_ContractDetails(), and Perl in IBAPI.pm's DESTROY method for ContractDetails. Hence, DISOWN() should be called for returned objects. Show quoted text

> However, the issue of segfaulting upon calling a non-existent SWIG > method is > not addressed by this.

I suspect that while the repro is slightly different, the underlying cause is the same. Try DISOWN()ing the $condet object before you call a nonexistent method (but really, just Don't Do That). I'm going to go with the "Doc, it hurts when I do this!" response to calling non-existent methods on SWIG-instantiated objects. There are already guards against it in SWIG.pm:AUTOLOAD(), and the SWIG-generated layer where possible. That said, patches would be welcome.

Sat Jul 12 01:46:06 2014 INFIDEL [...] cpan.org - Status changed from 'open' to 'resolved'