| Subject: | Performance improvement: Control array folding in XML::Simple::XMLout |
While using the module, I found that very large amounts of data passed
as content through to Solr were causing memory exhaustion. The cause
appeared to be the use of _fixXmlEndTag in HTTPUpdateHandler.pm to fix
up the XML generated by XML::Simple->XMLout.
The solution to my problem was to control array folding in XML::Simple
to avoid the fix up altogether. Patch attached to this ticket.
Hope this helps anyone running into similar issues.
Cheers
Vittal
--
Vittal Aithal
Cognidox Ltd
http://www.cognidox.com
| Subject: | solr-patch.txt |
--- lib/Solr/HTTPUpdateHandler.pm.orig 2008-02-12 21:41:35.000000000 +0000
+++ lib/Solr/HTTPUpdateHandler.pm 2009-12-15 17:19:08.925059380 +0000
@@ -46,7 +46,7 @@
foreach my $key ( keys %{$hash} ) {
# flush out hash ref to create solrXML format
- $addHash->{"field name=\"$key\""} = $hash->{$key};
+ $addHash->{"field"}->{$key} = { "content" => $hash->{$key} };
}
@@ -56,16 +56,13 @@
$xml .= XMLout(
$addHash,
NumericEscape => 2,
- ContentKey => 'name',
- NoAttr => 1,
+ NoAttr => 0,
+ KeyAttr => { field => 'name' },
RootName => 'doc',
);
$xml .= "</add>\n";
- # remove extra part of end string to fit solrXML format
- $xml = $self->_fixXmlEndTag($xml);
-
$self->_postRequest($xml, "add", $timeout);
return $self;