Bug #127814 for Lingua-EN-AddressParse: Parsing of UK towns is slow

Tue Nov 27 08:48:50 2018 NHORNE [...] cpan.org - Ticket created

Subject:

Parsing of UK towns is slow

Running some timing tests I've found that US and Canadian towns (not full addresses) is really quick - "Saskatoon, Saskatchewan, Canada" takes 0.1s, but of UK towns is slow - "Ramsgate, Kent, England" takes 1.07s. I realise that this package is about parsing addresses not towns, but do you have any pointers?

Tue Nov 27 08:51:55 2018 NHORNE [...] cpan.org - Correspondence added

To make it clear, the options I'm giving to new are: my $ap = Lingua::EN::AddressParse->new(country => 'GB', auto_clean => 1, force_case => 1, force_post_code => 0);

Tue Nov 27 08:54:00 2018 NHORNE [...] cpan.org - Correspondence added

Oh, and I just noticed something, the thing that's taking a long time isn't the call to parse(), it's actually the call to new() that takes a long time for GB.

Tue Nov 27 16:27:03 2018 kimryan [...] cpan.org - Correspondence added

Yes, the call to new is slow. But you should only need to do this once, and reuse the returned object for every address you parse. And never make the call to new from inside a loop.

Tue Nov 27 16:27:04 2018 The RT System itself - Status changed from 'new' to 'open'

Tue Nov 27 16:31:00 2018 NHORNE [...] cpan.org - Correspondence added

What I don't understand is why it's so much slower on GB data than the US and Canada.

Tue Nov 27 16:55:48 2018 kimryan [...] cpan.org - Correspondence added

I did a dump of the recurse-descent tree that is created. The UK one is twice as big as the US one, about 1.3 MB. Seems like the longer and more complex list of sub coutries in the UK causes the slowness compared to USA, Aust etc. It's really an issue for Parse::RecDescent module. But I think the setup time is reasonable for the complexity of the grammar.

Tue Nov 27 17:22:18 2018 kimryan [...] cpan.org - Correspondence added

Sorry, I've misread this so far, can see issue is with paring speed not setup time. I ran a test on some UK addresses and it processed 1000 records per hour. Are your addresses parsing without errors? The default pattern is to have a street number.

Tue Nov 27 17:24:18 2018 NHORNE [...] cpan.org - Correspondence added

Yes it does parse OK. I've re-factored my code to encapsulated parsers in my class (Geo::Coder::Free).

Tue Nov 27 17:27:49 2018 kimryan [...] cpan.org - Correspondence added

Can you give a few more data examples? I can only get UK addresses to parse with a street name and number.

Tue Nov 27 19:41:39 2018 NHORNE [...] cpan.org - Correspondence added

On Tue Nov 27 17:27:49 2018, KIMRYAN wrote: Show quoted text

> Can you give a few more data examples? I can only get UK addresses to > parse with a street name and number.

OK, so actually the parses fail so that's a read herring. I think you've answered what I need to know, which is all about instantiating objects and I now have a way forward. Thank you.

Tue Nov 27 19:42:00 2018 NHORNE [...] cpan.org - Status changed from 'open' to 'resolved'