Skip Menu |

This queue is for tickets about the XML-Compile CPAN distribution.

Report information
The Basics
Id: 120066
Status: resolved
Priority: 0/
Queue: XML-Compile

People
Owner: Nobody in particular
Requestors: SREZIC [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 1.54
Fixed in: 1.55



Subject: Add "json_friendly" option
It would be nice if it was possible to use XML::Compile for a "perfect" XML-to-json converter. In the current implementation some things are missing, e.g. the generated perl data structure does not know anything about booleans, so converting further to json looses this information. Also, some of the numeric XML values end up as strings in the json data structure. I created a patch with a new "json_friendly" option which fixes some of the issues. The patch can be found at <https://github.com/eserte/XML-Compile.forked/commit/7a14f041eff49df7419e0f67069f0d6e100a22c0> An overview of the changes: * add new option json_friendly * add two new builtin types: * boolean_with_Types_Serialiser is like boolean, but generates Types::Serialiser::Boolean values which seem to be understood by common json serializers (tried JSON::PP, JSON::XS, and Cpanel::JSON::XS) * sloppy_float_force_NV is like sloppy_float, but makes sure that the value is numified. This means trailing zeroes and a trailing dot got lost, which would break tests like 55facet.t (that's why I made another type) * a change to bigint(): the value is always numified. This does not break any test. * a new test t/92json.t, using the mentioned json serializers * no attempt was done for checking the other direction (i.e. json deserialization and xml writing)
Subject: Re: [rt.cpan.org #120066] Add "json_friendly" option
Date: Wed, 1 Feb 2017 21:39:36 +0100
To: Slaven_Rezic via RT <bug-XML-Compile [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* Slaven_Rezic via RT (bug-XML-Compile@rt.cpan.org) [170201 17:20]: Show quoted text
> Wed Feb 01 12:20:32 2017: Request 120066 was acted upon. > Transaction: Ticket created by SREZIC > Queue: XML-Compile > Subject: Add "json_friendly" option > Broken in: 1.54 > Severity: Wishlist > > It would be nice if it was possible to use XML::Compile for a "perfect" XML-to-json converter. > In the current implementation some things are missing, e.g. the > generated perl data structure does not know anything about booleans,
Of course, because Perl has no types: it an XML<->Perl translator. But I am not against some level of abuse ;-) Show quoted text
> An overview of the changes: > * add new option json_friendly
Couldn't that be 'typed_booleans', for instance? Other people wanted to know which values where actually booleans as well. It is not limited to JSON. Show quoted text
> * add two new builtin types: > * boolean_with_Types_Serialiser is like boolean, but generates > Types::Serialiser::Boolean values which seem to be understood > by common json serializers (tried JSON::PP, JSON::XS, and > Cpanel::JSON::XS)
Ugly class name. Why is it so badly named? Being boolean has nothing to do with the serialization. And I would like to avoid an additional dependency. But probably we have no other option :( It works because of this: *Types::Serialiser::Boolean:: = *JSON::PP::Boolean::; It's a pity that JSON::PP::Boolean itself is not overloaded. This looks somewhat complicated: + while(my($key, $val) = each %{$builtin_types{boolean}}) +{ + $builtin_types{boolean_with_Types_Serialiser}->{$key} = + $key eq 'parse' + ? sub { no warnings 'once'; + $_[0] =~ m/^\s*(false|0)\s*/i + ? $Types::Serialiser::false + : $Types::Serialiser::true } + : $val; +} Why not simply: $builtin_types{boolean_with_Types_Serialiser} = { %{$builtin_types{boolean}}, , parse => sub { no warnings 'once'; $_[0] =~ m/^\s*(false|0)\s*/i ? $Types::Serialiser::false : $Types::Serialiser::true }; It's a pity there is no $Type::Serialiser::bool($_[0] =~ m/^\s*(false|0)\s*/i); Show quoted text
> * sloppy_float_force_NV is like sloppy_float, but makes sure > that the value is numified. This means trailing zeroes and a > trailing dot got lost, which would break tests like 55facet.t > (that's why I made another type)
Is it really bad to have fail that test? (Which test is it?) It might be a good idea to fix the sloppy_float handler to numify: the user expects a float (not a string) and it is "sloppy" anyway ;-) Show quoted text
> * a change to bigint(): the value is always numified. This does > not break any test.
That's a correct fix. Show quoted text
> * a new test t/92json.t, using the mentioned json serializers
Show quoted text
> * no attempt was done for checking the other direction (i.e. json > deserialization and xml writing)
The XML schema already describes what the types are, so there should not be a problem... (I may need to change some ref's into reftype) -- MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
RT-Send-CC: Mark [...] Overmeer.net
On 2017-02-01 15:58:16, Mark@Overmeer.net wrote: Show quoted text
> * Slaven_Rezic via RT (bug-XML-Compile@rt.cpan.org) [170201 17:20]:
> > Wed Feb 01 12:20:32 2017: Request 120066 was acted upon. > > Transaction: Ticket created by SREZIC > > Queue: XML-Compile > > Subject: Add "json_friendly" option > > Broken in: 1.54 > > Severity: Wishlist > > > > It would be nice if it was possible to use XML::Compile for a > > "perfect" XML-to-json converter. > > In the current implementation some things are missing, e.g. the > > generated perl data structure does not know anything about booleans,
> > Of course, because Perl has no types: it an XML<->Perl translator. > But I am > not against some level of abuse ;-) >
> > An overview of the changes: > > * add new option json_friendly
> > Couldn't that be 'typed_booleans', for instance? Other people wanted > to > know which values where actually booleans as well. It is not limited > to > JSON.
There are also other attempts to get booleans into perl, but differently implemented. For example, there's boolean.pm, but it has no seamless integration with the JSON modules: $ perl -Mstrict -Mboolean -MJSON::MaybeXS -E 'say encode_json([true, false]);' encountered object '1', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at -e line 1. The boolean.pm documentation suggests to use the "convert_blessed" option: $ perl -Mstrict -Mboolean -MJSON::MaybeXS -E 'say JSON::MaybeXS->new->convert_blessed->encode([true, false]);' [true,false] In contrast, nothing special has to be done if using the special boolean types provided by the JSON modules or Types::Serialiser: $ perl -Mstrict -MJSON::MaybeXS -E 'say encode_json([JSON->true, JSON->false]);' [true,false] $ perl -Mstrict -MTypes::Serialiser -MJSON::MaybeXS -E 'say encode_json([$Types::Serialiser::true, $Types::Serialiser::false]);' [true,false] There's also a "Bool" in the Moose type system, but it seems that this is only about validation; there's no special class attached to such a value. Show quoted text
>
> > * add two new builtin types: > > * boolean_with_Types_Serialiser is like boolean, but generates > > Types::Serialiser::Boolean values which seem to be understood > > by common json serializers (tried JSON::PP, JSON::XS, and > > Cpanel::JSON::XS)
> > Ugly class name. Why is it so badly named? Being boolean has nothing > to > do with the serialization. And I would like to avoid an additional > dependency. But probably we have no other option :( > > It works because of this: > *Types::Serialiser::Boolean:: = *JSON::PP::Boolean::; > > It's a pity that JSON::PP::Boolean itself is not overloaded. > > This looks somewhat complicated: > > + while(my($key, $val) = each %{$builtin_types{boolean}}) > +{ > + $builtin_types{boolean_with_Types_Serialiser}->{$key} = > + $key eq 'parse' > + ? sub { no warnings 'once'; > + $_[0] =~ m/^\s*(false|0)\s*/i > + ? $Types::Serialiser::false > + : $Types::Serialiser::true } > + : $val; > +} > > Why not simply: > > $builtin_types{boolean_with_Types_Serialiser} = > { %{$builtin_types{boolean}}, > , parse => sub { > no warnings 'once'; > $_[0] =~ m/^\s*(false|0)\s*/i > ? $Types::Serialiser::false > : $Types::Serialiser::true > }; >
Yes, looks simpler. Show quoted text
> It's a pity there is no > > $Type::Serialiser::bool($_[0] =~ m/^\s*(false|0)\s*/i); >
> > * sloppy_float_force_NV is like sloppy_float, but makes sure > > that the value is numified. This means trailing zeroes and a > > trailing dot got lost, which would break tests like 55facet.t > > (that's why I made another type)
> > Is it really bad to have fail that test? (Which test is it?) > It might be a good idea to fix the sloppy_float handler to numify: the > user expects a float (not a string) and it is "sloppy" anyway ;-)
I think it was the following three: test_rw($schema, test12 => '<test12>1.10</test12>', '1.10'); test_rw($schema, test12 => '<test12>1.00</test12>', '1.00'); test_rw($schema, test12 => '<test12>1.</test12>', '1.'); If we numify here, then the information about trailing zeroes etc. is lost, and the roundtrip test fails. And there are situations where one want to like a perfect roundtrip. Show quoted text
> > * a change to bigint(): the value is always numified. This does > > not break any test.
> > That's a correct fix. >
> > * a new test t/92json.t, using the mentioned json serializers
>
> > * no attempt was done for checking the other direction (i.e. json > > deserialization and xml writing)
> > The XML schema already describes what the types are, so there should > not be > a problem... (I may need to change some ref's into reftype)
Regards, Slaven
RT-Send-CC: Mark [...] Overmeer.net
On 2017-02-02 05:03:26, SREZIC wrote: Show quoted text
> On 2017-02-01 15:58:16, Mark@Overmeer.net wrote:
[...] Show quoted text
> > > > This looks somewhat complicated: > > > > + while(my($key, $val) = each %{$builtin_types{boolean}}) > > +{ > > + $builtin_types{boolean_with_Types_Serialiser}->{$key} = > > + $key eq 'parse' > > + ? sub { no warnings 'once'; > > + $_[0] =~ m/^\s*(false|0)\s*/i > > + ? $Types::Serialiser::false > > + : $Types::Serialiser::true } > > + : $val; > > +} > > > > Why not simply: > > > > $builtin_types{boolean_with_Types_Serialiser} = > > { %{$builtin_types{boolean}}, > > , parse => sub { > > no warnings 'once'; > > $_[0] =~ m/^\s*(false|0)\s*/i > > ? $Types::Serialiser::false > > : $Types::Serialiser::true > > }; > >
> > Yes, looks simpler. >
I applied this suggestion as https://github.com/eserte/XML-Compile.forked/commit/d3ceb3d41befb2e3e11e9ebf3eb5cdc0ac0a40d2
Released half a year ago