Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the AnyEvent-Twitter-Stream CPAN distribution.

Report information
The Basics
Id: 55798
Status: open
Priority: 0/
Queue: AnyEvent-Twitter-Stream

People
Owner: Nobody in particular
Requestors: znmeb [...] cesmail.net
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.11
Fixed in: (no value)



Subject: Crashes with no traceback - suspect something is happening on Twitter end you aren't dealing with
I have a program that calls AnyEvent::Twitter::Stream to accumulate data from the "sample" stream. That program is attached ("Sample.pl"). Environment is openSUSE 11.2 64-bit - I can dump the whole environment details if you wish, but I think this is something Twitter is doing and not your code. Here's the error message I'm getting: ERROR: AnyEvent::Handle=HASH(0xd51738) at /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 102 runtime error in AnyEvent::guard callback: Can't call method "destroy" on an undefined value at /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 122. That's all I get - if there's a way to enable debugging so I can capture more info when this happens, let me know. This is very rare - I started this particular instance up at 2010-03-18 06:00:00 UTC and it crashed at 2010-03-22 13:29:35 UTC. Show quoted text
> perl -v
This is perl, v5.10.0 built for x86_64-linux-thread-multi Copyright 1987-2007, Larry Wall Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit. Complete documentation for Perl, including FAQ lists, should be found on this system using "man perl" or "perldoc perl". If you have access to the Internet, point your browser at http://www.perl.org/, the Perl Home Page. Show quoted text
> uname -a
Linux AlgoCompSynth 2.6.31.12-0.1-desktop #1 SMP PREEMPT 2010-01-27 08:20:11 +0100 x86_64 x86_64 x86_64 GNU/Linux I'm starting up another instance now.
Subject: Sample.pl
#!/usr/bin/perl -w use strict; use warnings; use Carp; use English qw(-no_match_vars); use AnyEvent::Twitter::Stream; use Sys::Statistics::Linux; #use IO::Compress::Bzip2 qw(bzip2 $Bzip2Error) ; # get a Linux stats handle my $lxs = Sys::Statistics::Linux->new( processes => { init => 1, pids => [ $$ ] } ); sleep 1; my $cumulative_cpu = 0; $cumulative_cpu = perfsnap($cumulative_cpu); my $auth = `cat znmeb_filter.auth`; my ($user, $password, $user_id) = split /:/, $auth; my $stamp = timestamp(); open my $STATUS, '>>:utf8', "${stamp}.json" or croak $OS_ERROR; my $done = AnyEvent->condvar; my $handle = AnyEvent::Twitter::Stream->new( username => $user, password => $password, method => 'sample', no_decode_json => 1, on_tweet => sub { # test for file rollover my $newstamp = timestamp(); if ($newstamp ne $stamp) { $cumulative_cpu = perfsnap($cumulative_cpu); close $STATUS or croak $OS_ERROR; system "bzip2 -9 ${stamp}.json &"; $stamp = $newstamp; open $STATUS, '>>:utf8', "${stamp}.json" or croak $OS_ERROR; } # route the JSON string to the filesystem my $tweet = shift; print {$STATUS} "${tweet}\n" or croak $OS_ERROR; }, on_error => sub { my $error = shift; carp "ERROR: $error"; $done->send; }, on_eof => sub { $done->send; }, ); $done->recv; sub timestamp { my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = gmtime time; my $timestamp = sprintf '%04d%02d%02d%02d', $year+1900, $mon+1, $mday, $hour; return $timestamp; } sub perfsnap { my ($cumulative_cpu) = @_; my $stats = $lxs->get; my $self = $stats->{processes}{$$}; my $cputime = $self->{ttime} + $cumulative_cpu; my ($days, $hours, $minutes, $seconds) = split /:/, $self->{actime}; my $walltime = 86400.0*$days + 3600.0*$hours + 60.0*$minutes + $seconds; my $ratio = $cputime/$walltime; printf STDERR "%.4f cpu, %.4f wall, %.4f ratio\n", $cputime, $walltime, $ratio; return $cputime; }
On Mon Mar 22 16:32:38 2010, znmeb wrote: Show quoted text
> I have a program that calls AnyEvent::Twitter::Stream to accumulate data > from the "sample" stream. That program is attached ("Sample.pl"). > Environment is openSUSE 11.2 64-bit - I can dump the whole environment > details if you wish, but I think this is something Twitter is doing and > not your code. Here's the error message I'm getting: > > ERROR: AnyEvent::Handle=HASH(0xd51738) at > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 102 > runtime error in AnyEvent::guard callback: Can't call method "destroy" > on an undefined value at > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 122.
Ed, whenever you're using AnyEvent, you need to make sure your event handlers don't die. That may be what's happening, here. I'd wrap the sub in an eval. I like Try::Tiny for that use Try::Tiny; on_tweet => sub { try { # test for file rollover my $newstamp = timestamp(); if ($newstamp ne $stamp) { $cumulative_cpu = perfsnap($cumulative_cpu); close $STATUS or croak $OS_ERROR; system "bzip2 -9 ${stamp}.json &"; $stamp = $newstamp; open $STATUS, '>>:utf8', "${stamp}.json" or croak $OS_ERROR; } # route the JSON string to the filesystem my $tweet = shift; print {$STATUS} "${tweet}\n" or croak $OS_ERROR; } catch { warn "unexpected error: $_"; }; },
CC: Marc Mims <marc [...] questright.com>
Subject: Re: [rt.cpan.org #55798] Crashes with no traceback - suspect something is happening on Twitter end you aren't dealing with
Date: Thu, 1 Apr 2010 13:43:58 -0700
To: bug-AnyEvent-Twitter-Stream [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Mon, Mar 22, 2010 at 1:32 PM, M. Edward (Ed) Borasky via RT <bug-AnyEvent-Twitter-Stream@rt.cpan.org> wrote: Show quoted text
> Mon Mar 22 16:32:38 2010: Request 55798 was acted upon.
Show quoted text
> ERROR: AnyEvent::Handle=HASH(0xd51738) at > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 102 > runtime error in AnyEvent::guard callback: Can't call method "destroy" > on an undefined value at > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 122.
This is caused by Twitter disconnecting you earlier and is due to my code trying to destroy handle even when it's already undef'ed by the error callback. I added a fix on the github master. Show quoted text
> > That's all I get - if there's a way to enable debugging so I can capture > more info when this happens, let me know. This is very rare - I started > this particular instance up at 2010-03-18 06:00:00 UTC and it crashed at > 2010-03-22 13:29:35 UTC. >
>> perl -v
> > This is perl, v5.10.0 built for x86_64-linux-thread-multi > > Copyright 1987-2007, Larry Wall > > Perl may be copied only under the terms of either the Artistic License > or the > GNU General Public License, which may be found in the Perl 5 source kit. > > Complete documentation for Perl, including FAQ lists, should be found on > this system using "man perl" or "perldoc perl".  If you have access to the > Internet, point your browser at http://www.perl.org/, the Perl Home Page. > >
>> uname -a
> Linux AlgoCompSynth 2.6.31.12-0.1-desktop #1 SMP PREEMPT 2010-01-27 > 08:20:11 +0100 x86_64 x86_64 x86_64 GNU/Linux > > I'm starting up another instance now. >
-- Tatsuhiko Miyagawa
From: znmeb [...] cesmail.net
On Thu Apr 01 16:44:15 2010, miyagawa@gmail.com wrote: Show quoted text
> On Mon, Mar 22, 2010 at 1:32 PM, M. Edward (Ed) Borasky via RT > <bug-AnyEvent-Twitter-Stream@rt.cpan.org> wrote:
> > Mon Mar 22 16:32:38 2010: Request 55798 was acted upon.
>
> > ERROR: AnyEvent::Handle=HASH(0xd51738) at > > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 102 > > runtime error in AnyEvent::guard callback: Can't call method "destroy" > > on an undefined value at > > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 122.
> > This is caused by Twitter disconnecting you earlier and is due to my > code trying to destroy handle even when it's already undef'ed by the > error callback. I added a fix on the github master.
Thanks! Is there any way I can get the HTML and TCP details when Twitter disconnects? I'd like to know what I'm doing wrong, so I can avoid triggering their disconnect. Here's their documentation: http://apiwiki.twitter.com/Streaming-API-Documentation If Twitter is disconnecting me, I must be doing something wrong. Could you look this over and see if there's anything obvious? I don't want to get banned for not doing it "their way". Show quoted text
> > > > That's all I get - if there's a way to enable debugging so I can capture > > more info when this happens, let me know. This is very rare - I started > > this particular instance up at 2010-03-18 06:00:00 UTC and it crashed at > > 2010-03-22 13:29:35 UTC. > >
> >> perl -v
> > > > This is perl, v5.10.0 built for x86_64-linux-thread-multi > > > > Copyright 1987-2007, Larry Wall > > > > Perl may be copied only under the terms of either the Artistic License > > or the > > GNU General Public License, which may be found in the Perl 5 source kit. > > > > Complete documentation for Perl, including FAQ lists, should be found on > > this system using "man perl" or "perldoc perl".  If you have access
to the Show quoted text
> > Internet, point your browser at http://www.perl.org/, the Perl Home
Page. Show quoted text
> > > >
> >> uname -a
> > Linux AlgoCompSynth 2.6.31.12-0.1-desktop #1 SMP PREEMPT 2010-01-27 > > 08:20:11 +0100 x86_64 x86_64 x86_64 GNU/Linux > > > > I'm starting up another instance now. > >
> > >
Subject: Re: [rt.cpan.org #55798] Crashes with no traceback - suspect something is happening on Twitter end you aren't dealing with
Date: Thu, 1 Apr 2010 13:56:07 -0700
To: bug-AnyEvent-Twitter-Stream [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Thu, Apr 1, 2010 at 1:51 PM, M. Edward (Ed) Borasky via RT <bug-AnyEvent-Twitter-Stream@rt.cpan.org> wrote: Show quoted text
>> > ERROR: AnyEvent::Handle=HASH(0xd51738) at >> > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 102 >> > runtime error in AnyEvent::guard callback: Can't call method "destroy" >> > on an undefined value at >> > /usr/lib/perl5/site_perl/5.10.0/AnyEvent/Twitter/Stream.pm line 122.
>> >> This is caused by Twitter disconnecting you earlier and is due to my >> code trying to destroy handle even when it's already undef'ed by the >> error callback. I added a fix on the github master.
> > Thanks! Is there any way I can get the HTML and TCP details when Twitter > disconnects?
In this particular case (ERROR:), i don't think there's any details provided by Twitter. It's basically a network level error that has nothing to do with your code. Show quoted text
> I'd like to know what I'm doing wrong, so I can avoid > triggering their disconnect. Here's their documentation: > > http://apiwiki.twitter.com/Streaming-API-Documentation > > If Twitter is disconnecting me, I must be doing something wrong. Could > you look this over and see if there's anything obvious? I don't want to > get banned for not doing it "their way".
on_error callback now should correctly get the error reason passed, so you can dump that information to know -- but when on_error is triggered, again it's an HTTP connection level error and not app-level error. Show quoted text
>> > >> > That's all I get - if there's a way to enable debugging so I can capture >> > more info when this happens, let me know. This is very rare - I started >> > this particular instance up at 2010-03-18 06:00:00 UTC and it crashed at >> > 2010-03-22 13:29:35 UTC. >> >
>> >> perl -v
>> > >> > This is perl, v5.10.0 built for x86_64-linux-thread-multi >> > >> > Copyright 1987-2007, Larry Wall >> > >> > Perl may be copied only under the terms of either the Artistic License >> > or the >> > GNU General Public License, which may be found in the Perl 5 source kit. >> > >> > Complete documentation for Perl, including FAQ lists, should be found on >> > this system using "man perl" or "perldoc perl".  If you have access
> to the
>> > Internet, point your browser at http://www.perl.org/, the Perl Home
> Page.
>> > >> >
>> >> uname -a
>> > Linux AlgoCompSynth 2.6.31.12-0.1-desktop #1 SMP PREEMPT 2010-01-27 >> > 08:20:11 +0100 x86_64 x86_64 x86_64 GNU/Linux >> > >> > I'm starting up another instance now. >> >
>> >> >>
> > > >
-- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #55798] Crashes with no traceback - suspect something is happening on Twitter end you aren't dealing with
Date: Thu, 1 Apr 2010 13:59:35 -0700
To: bug-AnyEvent-Twitter-Stream [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Thu, Apr 1, 2010 at 1:57 PM, miyagawa@gmail.com via RT <bug-AnyEvent-Twitter-Stream@rt.cpan.org> wrote: Show quoted text
> > In this particular case (ERROR:), i don't think there's any details > provided by Twitter. It's basically a network level error that has > nothing to do with your code. >
>> I'd like to know what I'm doing wrong, so I can avoid >> triggering their disconnect. Here's their documentation: >> >> http://apiwiki.twitter.com/Streaming-API-Documentation >> >> If Twitter is disconnecting me, I must be doing something wrong. Could >> you look this over and see if there's anything obvious? I don't want to >> get banned for not doing it "their way".
> > on_error callback now should correctly get the error reason passed, so > you can dump that information to know -- but when on_error is > triggered, again it's an HTTP connection level error and not app-level > error.
Correction: on_error callback is called when your auth username/password is wrong, for instance -- you can look at the code what type of error you'll get. But in the original ERROR: destroy handle case that's an HTTP connection level error and nothing to do about it in your code. I'll make a new release of AnyEvent::Twitter::Stream with this fix. Show quoted text
>>> > >>> > That's all I get - if there's a way to enable debugging so I can capture >>> > more info when this happens, let me know. This is very rare - I started >>> > this particular instance up at 2010-03-18 06:00:00 UTC and it crashed at >>> > 2010-03-22 13:29:35 UTC. >>> >
>>> >> perl -v
>>> > >>> > This is perl, v5.10.0 built for x86_64-linux-thread-multi >>> > >>> > Copyright 1987-2007, Larry Wall >>> > >>> > Perl may be copied only under the terms of either the Artistic License >>> > or the >>> > GNU General Public License, which may be found in the Perl 5 source kit. >>> > >>> > Complete documentation for Perl, including FAQ lists, should be found on >>> > this system using "man perl" or "perldoc perl".  If you have access
>> to the
>>> > Internet, point your browser at http://www.perl.org/, the Perl Home
>> Page.
>>> > >>> >
>>> >> uname -a
>>> > Linux AlgoCompSynth 2.6.31.12-0.1-desktop #1 SMP PREEMPT 2010-01-27 >>> > 08:20:11 +0100 x86_64 x86_64 x86_64 GNU/Linux >>> > >>> > I'm starting up another instance now. >>> >
>>> >>> >>>
>> >> >> >>
> > > > -- > Tatsuhiko Miyagawa > >
-- Tatsuhiko Miyagawa
From: znmeb [...] borasky-research.net
As far as I know this has been fixed. How do we mark it as closed?