Skip Menu |

This queue is for tickets about the Inline-Struct CPAN distribution.

Report information
The Basics
Id: 99198
Status: resolved
Worked: 2 hours (120 min)
Priority: 0/
Queue: Inline-Struct

People
Owner: ETJ [...] cpan.org
Requestors: heinz.knutzen [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Memory leak with multiple assignment to string member
Date: Sun, 28 Sep 2014 22:29:05 +0200
To: bug-Inline-Struct [...] rt.cpan.org
From: Heinz Knutzen <heinz.knutzen [...] gmail.com>
Define a struct with member of type "char *". If you assign a new string to this member, memory of the previous value isn't freed. Example program: ============================================ use strict; use warnings; use Inline C => <<'END', STRUCTS => 1; struct Foo { char *str; }; END my $count = shift; my $o = Inline::Struct::Foo->new(); my $str = 's'x100000; for (1..$count) { $o->str($str); } ============================================ Memory usage: $ /usr/bin/time -f%M perl inline-struct-multi-str.pl 10000 4764 $ /usr/bin/time -f%M perl inline-struct-multi-str.pl 100000 12524 $ /usr/bin/time -f%M perl inline-struct-multi-str.pl 1000000 90144 $ /usr/bin/time -f%M perl inline-struct-multi-str.pl 10000000 866324 Used versions: - Inline-Struct-0.11 - Perl v5.18.2
Subject: [rt.cpan.org #99198] Memory leak even for integer value
Date: Tue, 30 Sep 2014 21:10:44 +0200
To: bug-Inline-Struct [...] rt.cpan.org
From: Heinz Knutzen <heinz.knutzen [...] gmail.com>
The memory leaks even for a simple integer value. And surprisingly, the leak is nearly as large as with the long string from my previous example. ============================================ use strict; use warnings; use Inline C => <<'END', STRUCTS => 1; struct Foo { int i; }; END my $count = shift; my $o = Inline::Struct::Foo->new(); my $x; for (1..$count) { $o->i(42); } ============================================ Memory usage: $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 1000 3792 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 100000 12328 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 1000000 89948 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 10000000 866128
This is the same problem. I::S doesn't yet have special handling for SV *, but it will shortly.
Subject: Re: [rt.cpan.org #99198] Memory leak with multiple assignment to string member
Date: Mon, 13 Oct 2014 20:46:47 +0200
To: bug-Inline-Struct [...] rt.cpan.org
From: Heinz Knutzen <heinz.knutzen [...] gmail.com>
I am excited to see support for SV* values in Inline::Struct.
On reflection, this issue has nothing to do with SV* handling, since the typemap system will turn the SVs args into the appropriate C type. It is also nothing to do with making a big list with (1..$count), as I first suspected. Investigating further.
I ended up using this snippet: use strict; use warnings; use Devel::Gladiator qw(walk_arena arena_ref_counts arena_table); use Inline C => <<'END', structs => 1, force_build => 1, clean_after_build => 0; struct Foo { int i; }; /* struct Foo { char *str; }; */ END my $count = shift; my $o = Inline::Struct::Foo->new(); #my $str = 's'x100000; #use Data::Dumper; warn Dumper arena_ref_counts(); # prints counts keyed by class my %dump1 = map { ("$_" => $_) } walk_arena(); for (my $c = 1; $c <= $count; $c++) { # $o->str($str); $o->i(42); } my %dump2 = map { ("$_" => $_) } walk_arena(); delete $dump2{$_} for keys %dump1; use Devel::Peek; Dump \%dump2; I was mortalising the object refs made by the setter methods, but never freeing them. Now this is being done (as of commit 1b3925b85b36c33558ba7bfa3bdc417128a5f4f7). There is an inherent difficulty with trying to automatically manage char *, per your original report. The Inline::Struct class has no way of knowing automatically what allocated the area of memory, so you would need to manage this with your own typemap.
Due to a slight error, the relevant commit is now actually 1f779b6fd8e01f70240c9a1f1208dd610bec2766
Actually 1dd013f280e1863b95bdaa92b684f2a761c3e8f1.
Subject: [rt.cpan.org #99198] Still memory leak even for integer value
Date: Wed, 12 Nov 2014 21:21:09 +0100
To: bug-Inline-Struct [...] rt.cpan.org
From: Heinz Knutzen <heinz.knutzen [...] gmail.com>
There is still a memory leak with new version 0.14 and the same test program. ============================================ use strict; use warnings; use Inline C => <<'END', STRUCTS => 1; struct Foo { int i; }; END my $count = shift; my $o = Inline::Struct::Foo->new(); my $x; for (1..$count) { $o->i(42); } ============================================ $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 1000 3752 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 10000 4136 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 100000 8016 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 1000000 46828 $ /usr/bin/time -f%M perl inline-struct-multi-int.pl 10000000 434912 Used versions: - Inline-Struct-0.14 - Perl v5.18.2
I can confirm this with 0.15, using this snippet: /usr/bin/time -f%M perl -Mblib -MInline=C,'struct Foo {int i;};',structs,1 -e '$o = Inline::Struct::Foo->new; $o->i(42) for 1..shift' 50000
And then a quick correction to this: /usr/bin/time -f%M perl -Mblib -MInline=C,'struct Foo {int i;};',structs,1 -e '$o = Inline::Struct::Foo->new; my $count = shift; for (my $i = 1; $i <= $count; $i++) { $o->i(42) }' 50000 (not making a list but just counting)
I've used the following code, which calls the settor method once before snapshotting the arena: use Devel::Gladiator qw(walk_arena arena_ref_counts arena_table); use Inline C => 'struct Foo {int i;};', structs => 1; $o = Inline::Struct::Foo->new; my $count = 1000000; $o->i(42); %d1 = map { ("$_" => $_) } walk_arena(); for (my $i = 1; $i <= $count; $i++) { $o->i(42) } %d2 = map { $d1{$_} ? () : ("$_" => $_) } walk_arena(); use Devel::Peek; DumpArray values %d2; This showed no perl-side memory leaks. Using this code: use Inline C => 'struct Foo {int i;};', structs => 1; $o = Inline::Struct::Foo->new; my $count = 10; for (my $i = 1; $i <= $count; $i++) { $o->i(42) } Like so: valgrind perl -Mblib tf 2>&1|less valgrind shows with count of 10: 29,374 allocs, 19,983 frees With 20: 29,394 allocs, 20,003 frees So +10 calls -> 20 allocs, 10 frees. This was because of a non-mortalised SV hiding in grammar.pm, which is now fixed. With commit 50a3bf2de90509eb906a4ea21a9f8b7bcaacb5cf (and 0.16), this issue is now resolved.