Skip Menu |

This queue is for tickets about the RRD-Editor CPAN distribution.

Report information
The Basics
Id: 86596
Status: resolved
Priority: 0/
Queue: RRD-Editor

People
Owner: Nobody in particular
Requestors: rarylson [...] vialink.com.br
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Re: Error in RRD::Editor update method in MacOS
Date: Sun, 30 Jun 2013 17:06:23 -0300
To: bug-RRD-Editor [...] rt.cpan.org
From: Rarylson Freitas <rarylson [...] vialink.com.br>
One more information: root@RarylsonMac:rrdedit# uname -a Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 -- Rarylson Freitas Vialink Informatica Ltda On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> wrote: Show quoted text
> One more information: > > root@RarylsonMac:rrdedit# uname -a > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 >
Looks like some sort of issue with the rrd file. Could you post the output of $rrd->info() or (if the file is not too large) $rrd->dump() as that might help with diagnosis. On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote: Show quoted text
> One more information: > > root@RarylsonMac:rrdedit# uname -a > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May > 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 > > -- > Rarylson Freitas > Vialink Informatica Ltda > > > > On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> > wrote: >
> > One more information: > > > > root@RarylsonMac:rrdedit# uname -a > > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
> >
>
Subject: Re: [rt.cpan.org #86596] Re: Error in RRD::Editor update method in MacOS
Date: Sun, 30 Jun 2013 17:32:34 -0300
To: bug-RRD-Editor [...] rt.cpan.org
From: Rarylson Freitas <rarylson [...] vialink.com.br>
Of course, I edit the file to: #!/usr/bin/env perl use RRD::Editor; my $file = "test/ping_rta.rrd"; my $rrd = RRD::Editor->new(); $rrd->open($file); print $rrd->info(); print "\n"; $rrd->dump(); print "\n"; $rrd->close(); And: rarylson@RarylsonMac:rrdedit$ ./test.pl > output.txt the output.txt file was attached. -- Rarylson Freitas Vialink Informatica Ltda On 30/06/2013, at 17:18, "Doug Leith via RT" <bug-RRD-Editor@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=86596 > > > Looks like some sort of issue with the rrd file. Could you post the output of $rrd->info() or (if the file is not too large) $rrd->dump() as that might help with diagnosis. > > On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote:
>> One more information: >> >> root@RarylsonMac:rrdedit# uname -a >> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May >> 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 >> >> -- >> Rarylson Freitas >> Vialink Informatica Ltda >> >> >> >> On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> >> wrote: >>
>>> One more information: >>> >>> root@RarylsonMac:rrdedit# uname -a >>> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
>> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
>>>
>>
> > >
Of course,

I edit the file to:

#!/usr/bin/env perl

use RRD::Editor;

my $file = "test/ping_rta.rrd";

my $rrd = RRD::Editor->new();
$rrd->open($file);

print $rrd->info();
print "\n";
$rrd->dump();
print "\n";

$rrd->close();

And:

rarylson@RarylsonMac:rrdedit$ ./test.pl > output.txt

the output.txt file was attached.

Message body is not shown because sender requested not to inline it.


--
Rarylson FreitasVialink Informatica Ltda



On 30/06/2013, at 17:18, "Doug Leith via RT" <bug-RRD-Editor@rt.cpan.org> wrote:

Show quoted text
<URL: https://rt.cpan.org/Ticket/Display.html?id=86596 >

Looks like some sort of issue with the rrd file.   Could you post the output of $rrd->info() or (if the file is not too large) $rrd->dump() as that might help with diagnosis.

On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote:
One more information:

root@RarylsonMac:rrdedit# uname -a
Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May
1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64

--
Rarylson Freitas
Vialink Informatica Ltda



On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br>
wrote:

One more information:

root@RarylsonMac:rrdedit# uname -a
Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64






Subject: Re: [rt.cpan.org #86596] Error in RRD::Editor update method in MacOS
Date: Sun, 30 Jun 2013 17:39:35 -0300
To: bug-RRD-Editor [...] rt.cpan.org
From: Rarylson Freitas <rarylson [...] vialink.com.br>

Message body is not shown because it is too large.

Message body is not shown because it is too large.

Ok, it looks like what's happening is that $rrd->update() is updating the rrd file in place and then closing the file to make sure the update is properly flushed to disk. You're then calling $rrd->save() on a closed file, which generates the error. A workaround is just to omit the $rrd->save() call - the update will still be saved correctly to the file on disk. This in place updating is discussed in the documentation of the update() method. To satisfy yourself that updates have really been applied you can use $rrd->info() or $rrd->dump(). I'll also update RRD::Editor to flush to disk without closing and release a new version. On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote: Show quoted text
> One more information: > > root@RarylsonMac:rrdedit# uname -a > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May > 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 > > -- > Rarylson Freitas > Vialink Informatica Ltda > > > > On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> > wrote: >
> > One more information: > > > > root@RarylsonMac:rrdedit# uname -a > > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
> >
>
Subject: Re: [rt.cpan.org #86596] Error in RRD::Editor update method in MacOS
Date: Sun, 30 Jun 2013 19:14:03 -0300
To: bug-RRD-Editor [...] rt.cpan.org
From: Rarylson Freitas <rarylson [...] vialink.com.br>
You're right! My last test shows this behavior: #!/usr/bin/env perl # This test file shows the behavior explained by Doug Leith. # See: https://rt.cpan.org/Public/Bug/Display.html?id=86596 use RRD::Editor; use Data::Printer; my $file = "test/ping_rta.rrd"; my $rrd = RRD::Editor->new(); $rrd->open($file); # File is opened here... print $rrd->last() . "\n"; p $rrd; # We get the last update $rrd->update("N:2:2:2:2"); # We update the RRD print $rrd->last() . "\n"; p $rrd; # Note that in the second 'p $rrd', the file descriptor was closed. # Before: fd *RRD::Editor::$__ANONIO__ (read-only, layers: unix perlio), # After: fd *RRD::Editor::$__ANONIO__ (layers: ), # The object was updated. We know this because the last update is showed in the rrdinfo. `rrdtool info $file`; # The object also shows the updated info print $rrd->info() . "\n"; # Now, we'll get an error. This shows that the file descriptor was closed after the 'update'. $rrd->save(); $rrd->close(); I think that flush to disk after all update is less performative than flush only after the save operation. If we does a lot of updates, the performance is bad. However, it's intersting show a pretty error when the user call save and the file descriptor was closed. -- Rarylson Freitas Vialink Informatica Ltda On 30/06/2013, at 18:09, Doug Leith via RT <bug-RRD-Editor@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=86596 > > > Ok, it looks like what's happening is that $rrd->update() is updating the rrd file in place and then closing the file to make sure the update is properly flushed to disk. You're then calling $rrd->save() on a closed file, which generates the error. > > A workaround is just to omit the $rrd->save() call - the update will still be saved correctly to the file on disk. This in place updating is discussed in the documentation of the update() method. To satisfy yourself that updates have really been applied you can use $rrd->info() or $rrd->dump(). > > I'll also update RRD::Editor to flush to disk without closing and release a new version. > > On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote:
>> One more information: >> >> root@RarylsonMac:rrdedit# uname -a >> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May >> 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 >> >> -- >> Rarylson Freitas >> Vialink Informatica Ltda >> >> >> >> On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> >> wrote: >>
>>> One more information: >>> >>> root@RarylsonMac:rrdedit# uname -a >>> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
>> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
>>>
>>
> > >
Subject: Re: [rt.cpan.org #86596] Error in RRD::Editor update method in MacOS
Date: Sun, 30 Jun 2013 23:27:28 -0300
To: bug-RRD-Editor [...] rt.cpan.org
From: Rarylson Freitas <rarylson [...] vialink.com.br>
Adding some informations… I did your workaround add it works. However, like a suspected, insert many points flushing all the time in the file was very slow. I'm creating a command line tool for edit RRDs and my project uses RRD::Editor a lot (I love this perl module. Congratulations for your module!). But, as you can see, updating points in a RRD was the slow part: rarylson@RarylsonMac:rrdedit$ time ./rrdedit.pl resize-step-rra --file test/ping_rta.rrd --id 1 --tostep 3600 --with-step Creating new RRD structure New RRD structure created Generating points Points generated DEBUG: From start to this moment: 0.0638129711151123 Inserting new points Points inserted DEBUG: From 'insert points' to this moment: 6.63582110404968 RRD migrated. RRA step changed from 3600 to 3600 real 0m6.815s user 0m3.984s sys 0m0.694s I'll test others modules like RRDs and RRD::Tweak in the "update RRD" part. My project is in GitHub: https://github.com/rarylson/rrdedit On 30/06/2013, at 19:14, Rarylson Freitas <rarylson@vialink.com.br> wrote: Show quoted text
> You're right! > > My last test shows this behavior: > > #!/usr/bin/env perl > > # This test file shows the behavior explained by Doug Leith. > # See: https://rt.cpan.org/Public/Bug/Display.html?id=86596 > > use RRD::Editor; > > use Data::Printer; > > my $file = "test/ping_rta.rrd"; > > my $rrd = RRD::Editor->new(); > $rrd->open($file); > > # File is opened here... > > print $rrd->last() . "\n"; > p $rrd; > > # We get the last update > > $rrd->update("N:2:2:2:2"); > > # We update the RRD > > print $rrd->last() . "\n"; > p $rrd; > > # Note that in the second 'p $rrd', the file descriptor was closed. > # Before: fd *RRD::Editor::$__ANONIO__ (read-only, layers: unix perlio), > # After: fd *RRD::Editor::$__ANONIO__ (layers: ), > > # The object was updated. We know this because the last update is showed in the rrdinfo. > `rrdtool info $file`; > > # The object also shows the updated info > print $rrd->info() . "\n"; > > # Now, we'll get an error. This shows that the file descriptor was closed after the 'update'. > > $rrd->save(); > $rrd->close(); > > I think that flush to disk after all update is less performative than flush only after the save operation. If we does a lot of updates, the performance is bad. > > However, it's intersting show a pretty error when the user call save and the file descriptor was closed. > > -- > Rarylson Freitas > Vialink Informatica Ltda > > > On 30/06/2013, at 18:09, Doug Leith via RT <bug-RRD-Editor@rt.cpan.org> wrote: >
>> <URL: https://rt.cpan.org/Ticket/Display.html?id=86596 > >> >> Ok, it looks like what's happening is that $rrd->update() is updating the rrd file in place and then closing the file to make sure the update is properly flushed to disk. You're then calling $rrd->save() on a closed file, which generates the error. >> >> A workaround is just to omit the $rrd->save() call - the update will still be saved correctly to the file on disk. This in place updating is discussed in the documentation of the update() method. To satisfy yourself that updates have really been applied you can use $rrd->info() or $rrd->dump(). >> >> I'll also update RRD::Editor to flush to disk without closing and release a new version. >> >> On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote:
>>> One more information: >>> >>> root@RarylsonMac:rrdedit# uname -a >>> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May >>> 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 >>> >>> -- >>> Rarylson Freitas >>> Vialink Informatica Ltda >>> >>> >>> >>> On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> >>> wrote: >>>
>>>> One more information: >>>> >>>> root@RarylsonMac:rrdedit# uname -a >>>> Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
>>> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
>>>>
>>>
>> >> >>
>
I did testing against RRDtool and I found it to be slightly slower than perl for repeated updates. If you comment out line 1265 in Editor.pm (which closes the file after an update), and rerun you're tests you'll be able to confirm whether that is the source of the delay you're seeing. On Sun Jun 30 16:06:39 2013, rarylson@vialink.com.br wrote: Show quoted text
> One more information: > > root@RarylsonMac:rrdedit# uname -a > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May > 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64 > > -- > Rarylson Freitas > Vialink Informatica Ltda > > > > On 30/06/2013, at 17:05, Rarylson Freitas <rarylson@vialink.com.br> > wrote: >
> > One more information: > > > > root@RarylsonMac:rrdedit# uname -a > > Darwin RarylsonMac.local 12.4.0 Darwin Kernel Version 12.4.0: Wed
> May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
> >
>