Bug #61120 for CGI: redirect removes "&" and ";" from query strings

Mark, Show quoted text
HTML::Entities, probably the most widely used Module for escaping/unescaping, uses a list of named entities and their corresponding code points. Unrecognized entities are left alone. As I understand the function unescapeHTML, it knows 2 named entities, which it handles as intended, and numeric entities, which it replaces with their correspoding chr(). IMO a useful fix would be to keep this behavior, but leave everything else alone, without stripping the "&" and ";" as it currently does in these cases. However, and I am aware that this might probably open a can of worms, I was surprised that ->redirect() alters the supplied URL at all. This was not the behavior I expected. I do not know why this is done, maybe RFC-compliance or some other reason? I am not asking for changing this behavior, because it would break backwards compatibility, but I think that a) it should be documented and b) there should be a way to turn it off and use the URL "as is" - maybe with a flag in the method call, or a different method. This for both the HTML- und entity-escaping. Even if the reason is RFC-compliance, there are so many companies around that use and require messy query strings, that strictly following the RFC means trouble. For instance, I ran into this problem with URLs from a big german affiliate agency. I could work around that in my case, but that was pure chance. Karlheinz

I've looked into this bug further. The crux of it has to do with this line in sub redirect () {} return $self->header((map {$self->unescapeHTML($_)} @o),@unescaped); Some of the values passed to redirect() are being run through unescapeHTML. I'm stumped by this, because redirect() should not receiving HTML as input, nor does it generate HTML as output (it generates an HTTP header). It looks to me like the call to unescapeHTML() should not be there, and should be removed. I looked some at Mojo and HTTP::Headers, and neither of these do anything with HTML-escaping when generating headers. Further, I can't how the current behavior would be considered helping in working towards a spec. The unescapeHTML behavior was added in version 2.90 with the Changelog entry of simply "1. Fixed bug in redirect header handling.". Originally, it ran unescapeHTML on *all* values. Soon it was found that this behavior corrupted cookies, so it was partially backed-out in the 2.92 release, when it no longer applied too cookies. I think now we are seeing that it can also corrupt URLs in some cases, and should be removed. All that said, redirect() appears that it would not have an issue if there were also not a bug in how unescapeHTML handles "&foo;" sequences when it doesn't recognize the the entity name. Yanick, Lincoln, comments?