Skip Menu |

This queue is for tickets about the Tree-Suffix CPAN distribution.

Report information
The Basics
Id: 24865
Status: resolved
Priority: 0/
Queue: Tree-Suffix

People
Owner: Nobody in particular
Requestors: pust [...] isi.edu
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.14
Fixed in: (no value)



Subject: Memory Leak with search()
It seems that the search method in Tree::Suffix leaks memory. I have been unable to determine if the fault lies in the wrapper code, or one of the calls to libstree. I am using version 0.4.2 of libstree, and version 0.14 of Tree::Suffix. I am debugging on perl version "v5.8.6 built for darwin-thread-multi-2level", though the bug has also been confirmed on "v5.8.5 built for i386-linux-thread-multi" I have attached a small perl program that I hope will demonstrate the problem. Thanks for your work on this library, and please let me know if I can assist you in tracking down this bug. Michael Pust
Subject: TreeSuffixLeak.pl
#!/usr/bin/env perl use Tree::Suffix; my $str = "mississippi"; my $tree = Tree::Suffix->new($str); while (1) { my @matches = $tree->search("is"); }
From: pust [...] isi.edu
I believe now the bug is simply related to the creation of the answer array. I added a function called dummy_find to Suffix.xs: void dummy_find (self, string) Tree::Suffix self SV *string ALIAS: dummy_match = 1 dummy_search = 2 PROTOTYPE: $$ PREINIT: int len = 3; int i = 0; AV *match; PPCODE: /* just create nonsense match array, to isolate memory leak */ for(i = 0; i != len; ++i) { match = (AV *)sv_2mortal((SV *)newAV()); av_extend(match, 3); av_push(match, newSViv(0)); av_push(match, newSViv(i)); av_push(match, newSViv(i + 1)); XPUSHs(newRV_noinc((SV *)match)); } if (GIMME_V == G_SCALAR) XSRETURN_IV(SP - MARK); It uses the same procedure for building the return array, but makes no libstree calls. Then the script: #!/usr/bin/env perl use Tree::Suffix; my $str = "mississippi"; my $tree = Tree::Suffix->new($str, scalar reverse $str); while (1) { my @matches = $tree->dummy_search("is"); } still leaks memory. im not sure how all these av_push, sv_2mortal, newRV_noinc all work together to manage ref counts, but ill see if i can figure out a non-leaking solution
From: pust [...] isi.edu
I have a fix that works for me, though i still dont exactly know whats going on with the perlapi. I delay the sv_2mortal call to right before i push a new match array onto the return value. I am attaching a patch.
--- Tree-Suffix-0.14.old/Suffix.xs 2006-04-03 05:48:15.000000000 -0700 +++ Tree-Suffix-0.14/Suffix.xs 2007-02-10 10:06:48.000000000 -0800 @@ -239,12 +239,12 @@ find (self, string) while (node = stack.tqh_first) { TAILQ_REMOVE(&stack, stack.tqh_first, iteration); if (lst_node_is_leaf(node)) { - match = (AV *)sv_2mortal((SV *)newAV()); + match = (AV *)newAV(); av_extend(match, 3); av_push(match, newSViv(lst_stree_get_string_index(self, node->up_edge->range.string))); av_push(match, newSViv(node->index)); av_push(match, newSViv(node->index + len - 1)); - XPUSHs(newRV_noinc((SV *)match)); + XPUSHs(sv_2mortal(newRV_noinc((SV *)match))); } for (edge = node->kids.lh_first; edge; edge = edge->siblings.le_next) TAILQ_INSERT_HEAD(&stack, edge->dst_node, iteration);
From: jens.voeckler [...] gmail.com
I am a colleague of Mr. Pust. I've compiled his fixes, including a successful test, on Perl 5.8.5 as i386-linux-thread-multi and x86_64-linux-thread-multi platforms. However, when compiling for Perl 5.8.8 on x86_64-linux-thread-multi, the "make test" fails in test 09, sometimes with SIGBUS and sometimes with SIGSEGV. Looking at the core file, the offensive t/00_load............ok t/01_new.............ok t/02_insert..........ok t/03_lcs.............ok t/04_lrs.............ok t/05_clear...........ok t/06_remove..........ok t/07_memleak.........ok t/08_find............ok t/09_duplicates......dubious Test returned status 0 (wstat 135, 0x87) DIED. FAILED tests 2-13 Failed 12/13 tests, 7.69% okay t/10_pod.............ok t/11_pod_coverage....ok t/12_string..........ok Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- t/09_duplicates.t 0 135 13 24 2-13 Failed 1/13 test scripts. 12/59 subtests failed. Files=13, Tests=59, 2 wallclock secs ( 1.00 cusr + 0.28 csys = 1.28 CPU) Failed 1/13 test programs. 12/59 subtests failed. I detected a core file generated by a bus error: #0 XS_Tree__Suffix_remove (my_perl=Variable "my_perl" is not available.) at Suffix.xs:169 #1 0x0000002a955f351b in Perl_pp_entersub (my_perl=0x504010) at pp_hot.c:2877 #2 0x0000002a955d8422 in Perl_runops_debug (my_perl=0x504010) at dump.c:1459 #3 0x0000002a955932e4 in perl_run (my_perl=0x504010) at perl.c:2366 #4 0x00000000004019fc in main (argc=3, argv=0x7fbffff4c8, env=Variable "env" is not available.) at perlmain.c:99 The second time around was a SIGSEGV. (gdb) where #0 XS_Tree__Suffix_remove (my_perl=Variable "my_perl" is not available.) at Suffix.xs:169 #1 0x0000002a955f351b in Perl_pp_entersub (my_perl=0x504010) at pp_hot.c:2877 #2 0x0000002a955d8422 in Perl_runops_debug (my_perl=0x504010) at dump.c:1459 #3 0x0000002a955932e4 in perl_run (my_perl=0x504010) at perl.c:2366 #4 0x00000000004019fc in main (argc=3, argv=0x7fbffff4f8, env=Variable "env" is not available.) at perlmain.c:99 The offensive source line: 169 if (lst_string_get_length(hi->string) != len)
On Sat Feb 10 22:53:23 2007, voeckler wrote: Show quoted text
> I am a colleague of Mr. Pust. I've compiled his fixes, including a > successful test, on Perl 5.8.5 as i386-linux-thread-multi and > x86_64-linux-thread-multi platforms. > > However, when compiling for Perl 5.8.8 on x86_64-linux-thread-multi, > the > "make test" fails in test 09, sometimes with SIGBUS and sometimes with > SIGSEGV.
You fail to mention if `make test` fails without the patch. Note, I will not even be able to test the patch for at least another 2 weeks as I do not currently have access to a development machine.
fixed in 0.15