Bug #33166 for Locale-Maketext-Lexicon: Support in Extract.pm->extract for HTML::FormFu config-files

After testing some more, I found some issues with my last patch. It basicly mucks up if there are several lines :/ I have written some more testcases, and a new patch.

Only in Locale-Maketext-Lexicon-0.65: Makefile Only in Locale-Maketext-Lexicon-0.65: blib diff -ur Locale-Maketext-Lexicon-0.65.orig/lib/Locale/Maketext/Extract/Run.pm Locale-Maketext-Lexicon-0.65/lib/Locale/Maketext/Extract/Run.pm --- Locale-Maketext-Lexicon-0.65.orig/lib/Locale/Maketext/Extract/Run.pm 2008-02-12 11:28:47.000000000 +0100 +++ Locale-Maketext-Lexicon-0.65/lib/Locale/Maketext/Extract/Run.pm 2008-02-12 12:48:44.000000000 +0100 @@ -52,6 +52,7 @@ foreach my $file (@{$opts{f}||[]}) { open FILE, $file or die "Cannot open $file: $!"; while (<FILE>) { + chomp; push @ARGV, $_ if -r and !-d; } } diff -ur Locale-Maketext-Lexicon-0.65.orig/lib/Locale/Maketext/Extract.pm Locale-Maketext-Lexicon-0.65/lib/Locale/Maketext/Extract.pm --- Locale-Maketext-Lexicon-0.65.orig/lib/Locale/Maketext/Extract.pm 2008-02-12 11:28:48.000000000 +0100 +++ Locale-Maketext-Lexicon-0.65/lib/Locale/Maketext/Extract.pm 2008-02-12 15:05:10.000000000 +0100 @@ -51,6 +51,14 @@ Sentences between C<STARTxxx> and C<ENDxxx> are extracted individually. +=item HTML::FormFu + +HTML::FormFu uses a config-file to generate forms, with built in +support for localizing errors, labels etc. + +We extract the text after C<_loc: >: + content_loc: this is the string + =item Generic Template Strings inside {{...}} are extracted. @@ -234,6 +242,14 @@ $self->add_entry($str, [ $file, $line, $vars ]); } + # HTML::FormFu config-files + $line = 1; pos($_) = 0; + while (m/\G(.*?_loc:\s+(.*?)($|\n))/sg) { + my ($str) = $2; + $line += ( () = ($1 =~ /\n/g) ); # crypto something + $self->add_entry($str, [ $file, $line ]); + } + # Generic Template: $line = 1; pos($_) = 0; while (m/\G(.*?(?<!\{)\{\{(?!\{)(.*?)\}\})/sg) { Only in Locale-Maketext-Lexicon-0.65: pm_to_blib diff -ur Locale-Maketext-Lexicon-0.65.orig/t/5-extract.t Locale-Maketext-Lexicon-0.65/t/5-extract.t --- Locale-Maketext-Lexicon-0.65.orig/t/5-extract.t 2008-02-12 11:28:48.000000000 +0100 +++ Locale-Maketext-Lexicon-0.65/t/5-extract.t 2008-02-12 15:05:33.000000000 +0100 @@ -1,7 +1,7 @@ #! /usr/bin/perl -w use lib '../lib'; use strict; -use Test::More tests => 28; +use Test::More tests => 33; use_ok('Locale::Maketext::Extract'); my $Ext = Locale::Maketext::Extract->new; @@ -45,6 +45,13 @@ extract_ok(q(_(qq{foo\bar})) => "foo\bar", 'Interpolated \b in qq'); +# HTML::FormFu test +extract_ok(' content_loc: foo bar' => "foo bar", "html-formfu extraction"); +extract_ok(" content_loc: foo bar\n attributes: somethingelse" => 'foo bar', 'html-formfu extraction with newline'); +extract_ok(" content_loc: foo bar's" => "foo bar's", "html formfu extraction with quotes"); +extract_ok(" content_loc: foo\" bar", => 'foo" bar', "html formu extraciton with escaped quotes"); +extract_ok(" content_loc: foo bar\nvalue_loc: another_location" => 'foo baranother_location', "html formfu multiple extration"); + extract_ok( q(my $x = loc('I "think" you\'re a cow.') . "\n";) => 'I "think" you\'re a cow.', "Handle escaped single quotes"

I notice the HTML-FormFu support is still broken. Note that it isn't really specific to HTML-FormFu - this can be used for any YAML file that uses a hash-key ending in "_loc" to identify a L10N string. I've rewritten the support, so that instead of using a regex to try to parse YAML - it now uses YAML.pm and Data::Visitor::Callback. Any file whose first non-comment line is the yaml start-of-document marker "---" will be treated as a YAML file. Taking the existing code to handle Text::Template as a precedent - I've not added YAML or Data::Visitor::Callback to the Makefile.PL prereqs - the modules are only require()d if a YAML filetype is encountered. Though if you're OK with these modules being added to the prereqs - I'll happily provide an updated patch to do that, and to add tests. Carl

Index: t/5-extract.t =================================================================== --- t/5-extract.t (revision 1) +++ t/5-extract.t (working copy) @@ -1,7 +1,7 @@ #! /usr/bin/perl -w use lib '../lib'; use strict; -use Test::More tests => 43; +use Test::More tests => 42; use_ok('Locale::Maketext::Extract'); my $Ext = Locale::Maketext::Extract->new; @@ -44,9 +44,6 @@ extract_ok(q(_(q{foo\\\\bar})) => 'foo\\bar', 'Normalized \\\\ in q'); extract_ok(q(_(qq{foo\bar})) => "foo\bar", 'Interpolated \b in qq'); -# HTML::FormFu test -extract_ok(' content_loc: foo bar' => "foo bar", "html-formfu extraction"); - extract_ok( q(my $x = loc('I "think" you\'re a cow.') . "\n";) => 'I "think" you\'re a cow.', "Handle escaped single quotes" Index: lib/Locale/Maketext/Extract.pm =================================================================== --- lib/Locale/Maketext/Extract.pm (revision 1) +++ lib/Locale/Maketext/Extract.pm (working copy) @@ -244,6 +244,36 @@ $_ = ''; } + # YAML stream + my $yaml_doc = qr/ + ^ + (?: \s*? \# [^\n]*\n )* # any number of comment lines + ---\n # start of stream + /x; + + if ( $_ =~ $yaml_doc ) { + require YAML; + require Data::Visitor::Callback; + + my $data_visitor = Data::Visitor::Callback->new( + ignore_return_values => 1, + hash => sub { + my ( $visitor, $data ) = @_; + while ( my ( $key, $value ) = each %$data ) { + if ( $key =~ /_loc\z/ ) { + $self->add_entry( $value, [$file, $line] ); + } + } + }, + ); + + for my $document ( YAML::Load($_) ) { + $data_visitor->visit($document); + } + + $_ = ''; + } + # HTML::Mason $line = 1; pos($_) = 0; while (m!\G(.*?<&\|/l(?:oc)?(.*?)&>(.*?)</&>)!sg) { @@ -263,16 +293,8 @@ $trim_start && $str =~ s/^\s+//; $trim_end && $str =~ s/\s+$//; $self->add_entry($str, [ $file, $line, $vars ]); - } + } - # HTML::FormFu config-files - $line = 1; pos($_) = 0; - while (m/\G(.*?_loc:\s+(.*))/sg) { - my ($str) = $2; - $line += ( () = ($1 =~ /\n/g) ); # cryptocontext! - $self->add_entry($str, [ $file, $line ]); - } - # Generic Template: $line = 1; pos($_) = 0; while (m/\G(.*?(?<!\{)\{\{(?!\{)(.*?)\}\})/sg) {