Bug #79763 for perl-ldap: ldap_explode_dn issue with special characters

Thu Sep 20 11:54:19 2012 ctcard [...] hotmail.com - Ticket created

Subject:	ldap_explode_dn issue with special characters
Date:	Thu, 20 Sep 2012 15:54:08 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

According to the LDAP v3 specification, a DN like 'cn=CN\=#a' with an unescaped # character is a valid DN, but the function ldap_explode_dn in Net::LDAP::Util.pm rejects it. I am using Net::LDAP 0.40 (from perl-LDAP-0.40-1.el6.noarch) on centos 6, with perl 5.10.1, but as far as I can see this bug exists in all versions of Net::LDAP. This issue seems similar to one reported in the Spring LDAP parser (https://jira.springsource.org/browse/LDAP-229), and comments on that bug indicate that such DNs became valid in LDAP v3: "However there were made some significant DN syntax changes for LDAPv3 which are summarized at http://tools.ietf.org/html/rfc4514#appendix-B : + did not require escaping of equals sign ('=' U+003D) characters, + did not require escaping of non-leading number sign ('#' U+0023) characters, + allowed space (' ' U+0020) to be escaped as '\ ', + required hex escaping of null (U+0000) characters, and + removed LDAPv2-only constructs. "

Thu Sep 20 13:25:55 2012 peter [...] adpm.de - Correspondence added

CC:	ctcard [...] hotmail.com
Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Thu, 20 Sep 2012 19:30:36 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

Hi, On Thursday, 20. September 2012, Chris Card via RT wrote: Show quoted text

> Thu Sep 20 11:54:19 2012: Request 79763 was acted upon. > Transaction: Ticket created by ctcard@hotmail.com > Queue: perl-ldap > Subject: ldap_explode_dn issue with special characters > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: ctcard@hotmail.com > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=79763 > > > > > According to the LDAP v3 specification, a DN like 'cn=CN\=#a' with an > unescaped # character is a valid DN, but the function ldap_explode_dn in > Net::LDAP::Util.pm rejects it. > > I am using Net::LDAP 0.40 (from perl-LDAP-0.40-1.el6.noarch) on centos 6, > with perl 5.10.1, but as far as I can see this bug exists in all versions > of Net::LDAP. > > This issue seems similar to one reported in the Spring LDAP parser > (https://jira.springsource.org/browse/LDAP-229), and comments on that bug > indicate that such DNs became valid in LDAP v3: > > "However there were made some significant DN syntax changes for LDAPv3 > which are summarized at http://tools.ietf.org/html/rfc4514#appendix-B : + > did not require escaping of equals sign ('=' U+003D) characters, + did not > require escaping of non-leading number sign ('#' U+0023) characters, + > allowed space (' ' U+0020) to be escaped as '\ ', > + required hex escaping of null (U+0000) characters, and > + removed LDAPv2-only constructs. > "

You're right. all releases of perl-ldap up to 0.48 (which was released today) are affected. There is a patch in my repository at github that should cover most - if not all - of the changes: https://github.com/marschap/perl- ldap/commit/125632a514120ee72912e5fc5b5dd5d1e9c76075 Any feedback on this patch is very welcome. Best PEter -- Peter Marschall peter@adpm.de

Thu Sep 20 13:25:56 2012 The RT System itself - Status changed from 'new' to 'open'

Fri Sep 21 03:19:23 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 07:19:10 +0000
To:	<peter [...] adpm.de>, <bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Hi Peter, Show quoted text

> > According to the LDAP v3 specification, a DN like 'cn=CN\=#a' with an > > unescaped # character is a valid DN, but the function ldap_explode_dn in > > Net::LDAP::Util.pm rejects it.

<snip> Show quoted text

> You're right. > > all releases of perl-ldap up to 0.48 (which was released today) are affected. > > There is a patch in my repository at github that should cover most - if not > all - of the changes: > https://github.com/marschap/perl- > ldap/commit/125632a514120ee72912e5fc5b5dd5d1e9c76075 > > Any feedback on this patch is very welcome. > > Best > PEter

Thanks for that, I'll give the patch a try. I see you've made the regex much more readable if nothing else :) Is this patch going to make its way into a CPAN release any time soon? regards Chris

Fri Sep 21 03:54:37 2012 peter [...] adpm.de - Correspondence added

Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 09:59:29 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

Hi, On Friday, 21. September 2012, Chris Card via RT wrote: Show quoted text

> Thanks for that, I'll give the patch a try. I see you've made the regex > much more readable if nothing else :) > > Is this patch going to make its way into a CPAN release any time soon?

That depends on the feedback I get ;-) I don't want to release a broken implementation if I can help it. Especially if it affects an area of the code that is so central. Before your bug report came in, I got a bug report on the perl-ldap mailing list (that's why I was so quick with my answer about the patch ;-) where I am hoping for response too In addition to that, I wrote to the openldap-technical mailing list in the hope to get feedback from the "authorities" who co-authored the RFCs. You might want to run the test suite (t/01canon_dn.t) to see where the differences are between * the old implementation and the new one with rfc2253=>1 * the old implementation and the new one with rfc2253=>0 Best PEter -- Peter Marschall peter@adpm.de

Fri Sep 21 04:14:59 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 08:14:51 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Hi Show quoted text

> > Thanks for that, I'll give the patch a try. I see you've made the regex > > much more readable if nothing else :) > > > > Is this patch going to make its way into a CPAN release any time soon?

> > That depends on the feedback I get ;-) > I don't want to release a broken implementation if I can help it. > Especially if it affects an area of the code that is so central. > > Before your bug report came in, I got a bug report on the perl-ldap mailing > list (that's why I was so quick with my answer about the patch ;-) where I am > hoping for response too > > In addition to that, I wrote to the openldap-technical mailing list in the > hope to get feedback from the "authorities" who co-authored the RFCs. > > You might want to run the test suite (t/01canon_dn.t) to see where the > differences are between > * the old implementation and the new one with rfc2253=>1 > * the old implementation and the new one with rfc2253=>0

I downloaded the zip from your repository and did the usual perl Makefile.PL; make; make test stuff. I get 7 failures from t/01canon_dn.t: 1..70 ok 1 ok 2 ok 3 ok 4 ok 5 'OU=Sales+CN=J. Smith;O=Widget Inc.;C=US' failed to parse not ok 6 ok 7 ok 8 ok 9 'OU="Sales"+CN=J. Smith,O=Widget Inc.,C=US' failed to parse not ok 10 'OU="Sales+CN=J. Smith",O=Widget Inc.,C=US' failed to parse not ok 11 'cn=J.\20Smith\+ou=Sales,O=Widget\20Inc.,C=US' should not have parsed 'CN=J. Smith\2bou\3dSales,O=Widget Inc.,C=US' not ok 12 ok 13 <snipped> ok 24 'cn=Clif Harden+IDNumber="a0125589 ",ou=tiPerson,ou=person,o=ti,c=us' failed to parse not ok 25 ok 26 'Cn=" Graham Barr ",OU=person,O=vc,C=us' failed to parse not ok 27 'cn=" Graham \20Barr\20 ",OU=person,O=vc,C=us' failed to parse not ok 28 ok 29 <snipped> ok 70 Failed 7/70 subtests Is that expected? regards Chris

Fri Sep 21 04:50:13 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 08:49:59 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> Hi

> > > Thanks for that, I'll give the patch a try. I see you've made the regex > > > much more readable if nothing else :) > > > > > > Is this patch going to make its way into a CPAN release any time soon?

> > > > That depends on the feedback I get ;-) > > I don't want to release a broken implementation if I can help it. > > Especially if it affects an area of the code that is so central. > > > > Before your bug report came in, I got a bug report on the perl-ldap mailing > > list (that's why I was so quick with my answer about the patch ;-) where I am > > hoping for response too > > > > In addition to that, I wrote to the openldap-technical mailing list in the > > hope to get feedback from the "authorities" who co-authored the RFCs. > > > > You might want to run the test suite (t/01canon_dn.t) to see where the > > differences are between > > * the old implementation and the new one with rfc2253=>1 > > * the old implementation and the new one with rfc2253=>0

> I downloaded the zip from your repository and did the usual perl Makefile.PL; make; make test stuff. > I get 7 failures from t/01canon_dn.t: >

<snipped> Show quoted text

> Is that expected?

Answering my own question: yes, those failures are all for DNs which are valid in rfc2253 but not valid in rfc4514. However, I'm wondering if the patch is too strict, since openldap slapdn accepts DNs valid in rfc4514 or rfc2253. regards Chris

Fri Sep 21 05:41:39 2012 peter [...] adpm.de - Correspondence added

Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 11:46:29 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

On Friday, 21. September 2012, you wrote: Show quoted text

> Queue: perl-ldap > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=79763 > > [...]

> > Is that expected?

Yes, it is. the test suite is not updated yet. Show quoted text

> Answering my own question: yes, those failures are all for DNs which are > valid in rfc2253 but not valid in rfc4514. However, I'm wondering if the > patch is too strict, since openldap slapdn accepts DNs valid in rfc4514 or > rfc2253.

Hmmm, at a first glance that sounds a bit like wanting to keep the cake and eating it ;-) Changing the patch will definitely allow DNs which are illeagal in one of the two modes. (Remember: you wrote about "removed LDAPv2-only constructs" ;-) Nonetheless: ideas how the patch can modified to be more permissive are welcome. Best Peter -- Peter Marschall peter@adpm.de

Fri Sep 21 06:07:38 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 10:07:29 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> > Answering my own question: yes, those failures are all for DNs which are > > valid in rfc2253 but not valid in rfc4514. However, I'm wondering if the > > patch is too strict, since openldap slapdn accepts DNs valid in rfc4514 or > > rfc2253.

> Hmmm, at a first glance that sounds a bit like wanting to keep the cake and > eating it ;-) > Changing the patch will definitely allow DNs which are illeagal in one of the > two modes. > (Remember: you wrote about "removed LDAPv2-only constructs" ;-)

Ideally I'd like Net::LDAP to accept the same set of DNs as openldap, but I can work round that by calling ldap_explode_dn with rfc2253 set if a DN fails validation against the rfc4514 rules. Show quoted text

> > Nonetheless: ideas how the patch can modified to be more permissive are > welcome.

Chris

Fri Sep 21 06:30:58 2012 peter [...] adpm.de - Correspondence added

Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 12:35:44 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

On Friday, 21. September 2012, you wrote: Show quoted text

> > Ideally I'd like Net::LDAP to accept the same set of DNs as openldap,

Mission accomplished ;-) For those 7 DNs that fail the test suite (in one direction or the other), slapdn $DN behaves exactly as Net::LDAP::Util::ldap_explode_dn($DN, rfc2253 => 0) does Show quoted text

> but I > can work round that by calling ldap_explode_dn with rfc2253 set if a DN > fails validation against the rfc4514 rules. >

> > Nonetheless: ideas how the patch can modified to be more permissive are > > welcome.

I was hoping for patches ;-))) Best PEter -- Peter Marschall peter@adpm.de

Fri Sep 21 07:32:57 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 11:32:46 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> > > Ideally I'd like Net::LDAP to accept the same set of DNs as openldap,

> > Mission accomplished ;-) > For those 7 DNs that fail the test suite (in one direction or the other), > slapdn $DN > behaves exactly as > Net::LDAP::Util::ldap_explode_dn($DN, rfc2253 => 0) > does

Not for me (unless I've misunderstood what you mean). For example 'OU=Sales+CN=J. Smith;O=Widget Inc.;C=US' gives "failed to parse" in 01canon_dn.t, but slapdn accepts it (openldap 2.4.31). Chris

Fri Sep 21 11:59:45 2012 peter [...] adpm.de - Correspondence added

Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 18:04:35 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

Hi, On Friday, 21. September 2012, you wrote: Show quoted text

> Not for me (unless I've misunderstood what you mean). > For example 'OU=Sales+CN=J. Smith;O=Widget Inc.;C=US' gives "failed to > parse" in 01canon_dn.t, but slapdn accepts it (openldap 2.4.31).

Please also have a look at a new alternative solution to the ldap_explode_dn issue w.r.t. RFC 4514: https://github.com/marschap/perl-ldap/tree/ldap_explode_dn-relaxed This alternative tries to adhere to Postel's law, "be conservative in what you do, be liberal in what you accept from others". Pro's: - no additional option (like 'rfc2253') necessary - should not break existing code that relied on some RFC 2253 peculiarities (at least, the probability to do so is much lower) - higher compatibilitiy to openLDAP's slapdn than the other alternative Con's: - might accept DN's that are not conforming to any of the RFCs (but canonical_dn will make them confiorming to both) Which of the alternatives do you prefer? I guess this one is more to your liking ,-) Best PEter -- Peter Marschall peter@adpm.de

Fri Sep 21 12:21:28 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Fri, 21 Sep 2012 16:21:16 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> On Friday, 21. September 2012, you wrote:

> > Not for me (unless I've misunderstood what you mean). > > For example 'OU=Sales+CN=J. Smith;O=Widget Inc.;C=US' gives "failed to > > parse" in 01canon_dn.t, but slapdn accepts it (openldap 2.4.31).

> > Please also have a look at a new alternative solution to the ldap_explode_dn > issue w.r.t. RFC 4514: > https://github.com/marschap/perl-ldap/tree/ldap_explode_dn-relaxed > > This alternative tries to adhere to Postel's law, "be conservative in what you > do, be liberal in what you accept from others". > > Pro's: > - no additional option (like 'rfc2253') necessary > - should not break existing code that relied on some RFC 2253 peculiarities > (at least, the probability to do so is much lower) > - higher compatibilitiy to openLDAP's slapdn than the other alternative > > Con's: > - might accept DN's that are not conforming to any of the RFCs > (but canonical_dn will make them confiorming to both) > > Which of the alternatives do you prefer? > > I guess this one is more to your liking ,-)

Thanks! I'll give this version a try and get back to you (though it may not be until Monday now) Chris

Mon Sep 24 03:22:36 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Mon, 24 Sep 2012 07:22:16 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> > On Friday, 21. September 2012, you wrote:

> > > Not for me (unless I've misunderstood what you mean). > > > For example 'OU=Sales+CN=J. Smith;O=Widget Inc.;C=US' gives "failed to > > > parse" in 01canon_dn.t, but slapdn accepts it (openldap 2.4.31).

> > > > Please also have a look at a new alternative solution to the ldap_explode_dn > > issue w.r.t. RFC 4514: > > https://github.com/marschap/perl-ldap/tree/ldap_explode_dn-relaxed > > > > This alternative tries to adhere to Postel's law, "be conservative in what you > > do, be liberal in what you accept from others". > > > > Pro's: > > - no additional option (like 'rfc2253') necessary > > - should not break existing code that relied on some RFC 2253 peculiarities > > (at least, the probability to do so is much lower) > > - higher compatibilitiy to openLDAP's slapdn than the other alternative > > > > Con's: > > - might accept DN's that are not conforming to any of the RFCs > > (but canonical_dn will make them confiorming to both) > > > > Which of the alternatives do you prefer? > > > > I guess this one is more to your liking ,-)

> Thanks! I'll give this version a try and get back to you (though it may not be until Monday now)

That patch looks good to me, and yes, I would prefer it. Chris

Wed Oct 03 11:11:01 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Wed, 3 Oct 2012 15:10:51 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Hi Show quoted text

> Please also have a look at a new alternative solution to the ldap_explode_dn > issue w.r.t. RFC 4514: > https://github.com/marschap/perl-ldap/tree/ldap_explode_dn-relaxed > > This alternative tries to adhere to Postel's law, "be conservative in what you > do, be liberal in what you accept from others".

Any idea when this patch will be generally available? Chris

Wed Oct 03 11:42:47 2012 peter [...] adpm.de - Correspondence added

Subject:	Re: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Wed, 3 Oct 2012 17:48:03 +0200
To:	bug-perl-ldap [...] rt.cpan.org
From:	Peter Marschall <peter [...] adpm.de>

Hi, On Wednesday, 3. October 2012, you wrote: Show quoted text

> Any idea when this patch will be generally available?

I plan to release perl-ldap 0.49 in October, and it will contain the patch. Best Peter -- Peter Marschall peter@adpm.de

Thu Oct 04 05:00:48 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] ldap_explode_dn issue with special characters
Date:	Thu, 4 Oct 2012 09:00:26 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Show quoted text

> On Wednesday, 3. October 2012, you wrote:

> > Any idea when this patch will be generally available?

> > I plan to release perl-ldap 0.49 in October, and it will contain the patch. >

excellent, thanks! Chris

Sat Oct 06 10:08:23 2012 MARSCHAP [...] cpan.org - Fixed in 0.49 added

Sat Oct 06 10:09:14 2012 MARSCHAP [...] cpan.org - Correspondence added

Fixed in release 0.49 with the patch previously posted on github.

Sat Oct 06 10:09:15 2012 MARSCHAP [...] cpan.org - Status changed from 'open' to 'resolved'

Wed Oct 10 10:25:07 2012 ctcard [...] hotmail.com - Correspondence added

Subject:	RE: [rt.cpan.org #79763] Resolved: ldap_explode_dn issue with special characters
Date:	Wed, 10 Oct 2012 14:24:53 +0000
To:	<bug-perl-ldap [...] rt.cpan.org>
From:	Chris Card <ctcard [...] hotmail.com>

Hi ---------------------------------------- Show quoted text

> Subject: [rt.cpan.org #79763] Resolved: ldap_explode_dn issue with special characters > From: bug-perl-ldap@rt.cpan.org > To: ctcard@hotmail.com > Date: Sat, 6 Oct 2012 10:09:15 -0400 > > <URL: https://rt.cpan.org/Ticket/Display.html?id=79763 > > > According to our records, your request has been resolved. If you have any > further questions or concerns, please respond to this message.

I'm trying to integrate Net::LDAP 0.49 into our environment, but I notice that the version of Net::LDAP::Util.pm has not changed - it's still at version 0.12 - which is causing me problems, since the fix is in that module. Chris

Wed Oct 10 10:25:08 2012 The RT System itself - Status changed from 'resolved' to 'open'

Sat Nov 24 10:03:07 2012 MARSCHAP [...] cpan.org - Status changed from 'open' to 'resolved'

Sat Nov 24 10:03:13 2012 MARSCHAP [...] cpan.org - Fixed in 0.50 added

Sat Nov 24 10:03:13 2012 MARSCHAP [...] cpan.org - Fixed in 0.49 deleted