Bug #108004 for JSON-Tiny: Large Ints End Up As Floats

Tue Oct 27 11:06:10 2015 cpan [...] zoffix.com - Ticket created

Subject:

Large Ints End Up As Floats

Hey, This was reported in Mojo::JSON (https://github.com/kraih/mojo/issues/859) and in my testing I noticed JSON::Tiny also has this issue. Not sure if you care, but I figured I'd mention it: As you can see here, if a JSON integer is large enough, it gets converted to a float. Many other JSON modules correctly handle this but JSON::Tiny (last) decode/encode operation doesn't result in the original string. zoffix@:~$ perl -MJSON::Meth -wlE 'say "[123456789012345678]"->$j->$j' ["123456789012345678"] zoffix@:~$ perl -MJSON::MaybeXS -wlE 'say encode_json decode_json "[123456789012345678]"' ["123456789012345678"] zoffix@:~$ perl -MJSON::PP -wlE 'say encode_json decode_json "[123456789012345678]"' ["123456789012345678"] zoffix@:~$ perl -MJSON::Parse=parse_json -MJSON::Create=create_json -wlE 'say create_json parse_json "[123456789012345678]"' ["123456789012345678"] zoffix@:~$ perl -MJSON -wlE 'say encode_json decode_json "[123456789012345678]"' ["123456789012345678"] zoffix@:~$ perl -MJSON::Tiny=j -wlE 'say j j "[123456789012345678]"' [1.23456789012346e+17] zoffix@:~$ -- Cheers, ZZ [ https://metacpan.org/author/ZOFFIX ]

Tue Oct 27 11:09:31 2015 cpan [...] zoffix.com - Correspondence added

Forgot to mention, I'm on 32-bit Perl on Debian: $ perl -V Summary of my perl5 (revision 5 version 22 subversion 0) configuration: Platform: osname=linux, osvers=3.2.0-4-686-pae, archname=i686-linux uname='linux b--dt 3.2.0-4-686-pae #1 smp debian 3.2.51-1 i686 gnulinux ' config_args='-de -Dprefix=/home/zoffix/perl5/perlbrew/perls/perl-5.22.0 -Aeval:scriptdir=/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/bin' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2', optimize='-O2', cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.7.2', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12, longdblkind=3 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/i486-linux-gnu/4.7/include-fixed /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: Devel::PatchPerl 1.30 Built under linux Compiled at Aug 12 2015 16:48:28 %ENV: PERLBREW_BASHRC_VERSION="0.73" PERLBREW_HOME="/home/zoffix/.perlbrew" PERLBREW_MANPATH="/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/man" PERLBREW_PATH="/home/zoffix/perl5/perlbrew/bin:/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/bin" PERLBREW_PERL="perl-5.22.0" PERLBREW_ROOT="/home/zoffix/perl5/perlbrew" PERLBREW_VERSION="0.73" @INC: /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/site_perl/5.22.0/i686-linux /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/site_perl/5.22.0 /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/5.22.0/i686-linux /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/5.22.0 . -- Cheers, ZZ [ https://metacpan.org/author/ZOFFIX ]

Tue Oct 27 11:13:00 2015 daoswald [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #108004] Large Ints End Up As Floats
Date:	Tue, 27 Oct 2015 09:12:34 -0600
To:	bug-JSON-Tiny [...] rt.cpan.org
From:	David Oswald <daoswald [...] gmail.com>

Thanks for the report. I noticed the thread on the mojo list, and saw that the solution is to handle capturing a little differently, to avoid $1. I'll get it fixed sometime this week. On Tue, Oct 27, 2015 at 9:09 AM, Zoffix Znet via RT < bug-JSON-Tiny@rt.cpan.org> wrote: Show quoted text

> Queue: JSON-Tiny > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=108004 > > > Forgot to mention, I'm on 32-bit Perl on Debian: > > $ perl -V > Summary of my perl5 (revision 5 version 22 subversion 0) configuration: > > Platform: > osname=linux, osvers=3.2.0-4-686-pae, archname=i686-linux > uname='linux b--dt 3.2.0-4-686-pae #1 smp debian 3.2.51-1 i686 > gnulinux ' > config_args='-de > -Dprefix=/home/zoffix/perl5/perlbrew/perls/perl-5.22.0 > -Aeval:scriptdir=/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/bin' > hint=recommended, useposix=true, d_sigaction=define > useithreads=undef, usemultiplicity=undef > use64bitint=undef, use64bitall=undef, uselongdouble=undef > usemymalloc=n, bincompat5005=undef > Compiler: > cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe > -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2', > optimize='-O2', > cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector > -I/usr/local/include' > ccversion='', gccversion='4.7.2', gccosandvers='' > intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234, > doublekind=3 > d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12, > longdblkind=3 > ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', > lseeksize=8 > alignbytes=4, prototype=define > Linker and Libraries: > ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' > libpth=/usr/local/lib /usr/lib/gcc/i486-linux-gnu/4.7/include-fixed > /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib > /usr/lib/i386-linux-gnu /usr/lib/../lib /lib > libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc > -lgdbm_compat > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > libc=libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a > gnulibc_version='2.13' > Dynamic Linking: > dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' > cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib > -fstack-protector' > > > Characteristics of this binary (from libperl): > Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV > PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP > PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV > USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE > USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME > USE_PERLIO USE_PERL_ATOF > Locally applied patches: > Devel::PatchPerl 1.30 > Built under linux > Compiled at Aug 12 2015 16:48:28 > %ENV: > PERLBREW_BASHRC_VERSION="0.73" > PERLBREW_HOME="/home/zoffix/.perlbrew" > PERLBREW_MANPATH="/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/man" > > PERLBREW_PATH="/home/zoffix/perl5/perlbrew/bin:/home/zoffix/perl5/perlbrew/perls/perl-5.22.0/bin" > PERLBREW_PERL="perl-5.22.0" > PERLBREW_ROOT="/home/zoffix/perl5/perlbrew" > PERLBREW_VERSION="0.73" > @INC: > > /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/site_perl/5.22.0/i686-linux > /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/site_perl/5.22.0 > /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/5.22.0/i686-linux > /home/zoffix/perl5/perlbrew/perls/perl-5.22.0/lib/5.22.0 > . > > > > -- > > Cheers, > ZZ [ https://metacpan.org/author/ZOFFIX ] >

-- David Oswald daoswald@gmail.com

Tue Oct 27 11:13:00 2015 The RT System itself - Status changed from 'new' to 'open'

Tue Oct 27 15:52:17 2015 cpan [...] zoffix.com - Correspondence added

I think we later found that solution doesn't work all the time. Also, something came to light that you may want to consider before fixing anything: the other modules mentioned actually end up encoding the number as a string, while JSON::Tiny encodes it as a number. That might matter more than quadrillionths percentage point of precision. -- Cheers, ZZ [ https://metacpan.org/author/ZOFFIX ]

Tue Oct 27 21:46:24 2015 davido [...] cpan.org - Correspondence added

On Tue Oct 27 12:52:17 2015, ZOFFIX wrote: Show quoted text

> I think we later found that solution doesn't work all the time. > > Also, something came to light that you may want to consider before > fixing anything: the other modules mentioned actually end up encoding > the number as a string, while JSON::Tiny encodes it as a number. That > might matter more than quadrillionths percentage point of precision.

Encoding as a number was enforced by the return 0+$i;, which has been intentional behavior. It may or may not be desirable, and I know that Mojo::JSON has gone back and forth on when to favor numeric encoding vs stringification. If we've become out of sync with Mojo::JSON we could probably raise that as a different issue. Meanwhile, I've fixed the original problem by applying a patch similar to what was suggested for Mojo::JSON: my ($i) = m/....../gc; return 0+$i if defined $i;

Tue Oct 27 21:48:15 2015 davido [...] cpan.org - Correspondence added

Resolving the original issue of precision loss for large ints in release 0.54.

Tue Oct 27 21:48:21 2015 davido [...] cpan.org - Status changed from 'open' to 'resolved'