Le 2012-09-11 19:18:09, SMUELLER a écrit :
Show quoted text> Interesting. Well, within a given Sereal packet, we will never achieve
> "proper" streaming.
That's not what I'm asking for.
Show quoted text> Our back-reference logic may requires being able to
> jump forward and backward within the string buffer that holds the Sereal
> packet.
That is fine. At least the serialisation protocol should allows a
decoder to be able to detect packet boundaries without relying only on
the size of the input buffer.
Is Sereal clean on that matter?
Show quoted text> But what's possible (already now) is concatenating multiple
> Sereal packets and then while(){} ing over them. To wit from the docs
> (with a bug fix):
>
> my @out;
> my $pos = 0;
> eval {
> while (1) {
> push @out, $decoder->decode_with_offset($sereal_string, 0);
> $pos += $decoder->bytes_consumed;
> last if $pos >= length($sereal_string)
> or not $decoder->bytes_consumed;
> }
> };
What I'm asking for is for a stateful decoder that works when
$sereal_string is not a complete Sereal buffer and that, when fed, can
report if it got too few or too much data. When it has a full packet it
can return it decoded.
This could allow to put in a binary stream various packets encoded using
different encoders (the application writing/reading the stream is
responsible for calling successively each decoder as appropriate).
AnyEvent supports this with JSON (thanks to JSON::XS) has this capability.
MessagePack has some partial support (Data::MessagePack::Stream), but it
is incomplete (it doesn't report the consumed bytes, so MessagePack
packets can't be mixed with other packets).
--
Olivier Mengué -
http://perlresume.org/DOLMEN
