| Subject: | ExtendedEncoding normalizes incorrectly |
I believe ExtendedEncoding.pm normalizes incorrectly; when it constrains
values, it constrains them to between 0 and $self->max_value, when it
should constrain them between 0 and $scale. I think there is also an
off-by one error in the normalization; $scale is set to 4095 (64 * 64 -
1) but the constraint subtracts one again so the max value is 4094. This
may explain at least some cases of the "max value being encoded as ._
instead of .." issue.
Attached is a patch that addresses both issues.
--
--kulp
| Subject: | ExtendedEncoding.pm.diff |
--- /tmp/ExtendedEncoding.pm.old 2008-05-22 17:01:56.000000000 -0500
+++ /tmp/ExtendedEncoding.pm.new 2008-05-22 17:03:39.000000000 -0500
@@ -34,8 +34,8 @@
# Truncate values out of [0 .. max_value]
$normalized_value = 0
if $normalized_value <= 0;
- $normalized_value = $self->max_value - 1
- if $normalized_value >= $self->max_value;
+ $normalized_value = $scale
+ if $normalized_value > $scale;
my $hi = int($normalized_value / scalar(@map));
my $low = int($normalized_value % scalar(@map));