| Subject: | FB_QUIET is useless for error detection while nonblocking |
Setting CHECK to FB_DEFAULT, FB_CROAK or FB_WARN give you error-handling semantics of expected-
complete UTF-8 bytestrings.
Setting CHECK to FB_RETURN makes it immediately return on any failure.
None of these distinguish failure because an invalid byte was found, from failure due to running out
of bytes while processing a hitherto-valid encoding.
This makes it impossible to actually apply error-handling semantics to nonblocking or fixed-size
buffer reading, as your example otherwise would suggest.
I would recommend adding a new control bit, RETURN_ON_EOS, which has the behaviour of returning,
rather than signalling an error, but *only* if the failure was running out of bytes. This could be
applied in addition to the other control modes.
For example, the following would croak on actually-invalid input, but still correctly handle partial
UTF-8 encodings split across multiple reads:
my $buffer = ''; my $string = '';
while(read $fh, $buffer, 256, length($buffer)){
$string .= decode($encoding, $buffer, Encode::FB_CROAK|Encode::RETURN_ON_EOS);
# $buffer now contains the unprocessed partial character
}
--
Paul Evans